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ABSTRACT 

Details  of  the  design  and  data  analysis  associated 
with  the  1983  National  Assessment  of  Educational  Progress  (NAEP)  are 
presented.  The  1988  NAEP  surveyed  American  students'  knowledge  of 
reading,  writing,  civics,  U.S.  history,  and  geography.  Small-scale 
studies  were  conducted  for  mathematics  and  science.  Populations 
included  public  school  students  and  private  school  students  (aged  9, 
13,  and  17  years)  as  well  as  students  in  grades  4,  8,  and  12. 
Geography  was  surveyed  only  at  grade  12/age  17  years.  The  objective 
of  these  ter  jiical  notes  is  to  provide  information  to  alio,  the 
reader  to  judge  the  utility  of  the  design,  quality  of  the  NAEP  data, 
reasonableness  of  the  assumptions  made,  appropriateness  of  the  data 
analyses,  and  generalizability  of  the  inferences  made  from  the  data. 
Topics  addressed  inclade:  development  of  objectives  and  items;  sample 
design;  assessment  instruments;  field  administration;  materials 
processing  and  database  creation;  processing  assessment  materials; 
professional  scoring;  data  transcription  systems;  editing  data; 
quality  control  of  data  entry;  database  products;  weighting 
procedures  and  estimation  of  sampling  variance;  scaling  procedures; 
data  analysis  for  the  various  subject  area  assessments;  and  the 
statistical  summary  'of  the  sarples  and  estimates  of  proficiencies  of 
U.S.  students.  A  total  of  204  data  tables  and  14  figures  are 
provided.  Appendices  provide:  a  list  of  consultants  for  development 
of  the  1988  NAEP  objectives  and  items;  distributions  of  weight 
components  for  the  1988  NAEP  samples;  contrast  codings  and  estimated 
effects  for  1988  NAEP  conditioning  variables;  1988  NAEP  derived  and 
composite  conditioning  variables;  revision  of  1984  NAEP 
post-stratification  weights  for  grade  4/age  9  years  and  grade  8/age 
13  years;  1988  NAEP  item  response  theory  parameters;  and  NAEP 
reporting  subgroups,  composite  and  derived  common  background 
variables,  and  subject-specific  composite  and  derived  reporting 
variables.  A  glossary  of  terms,  a  list  of  references  cited  in  the 
text/  and  an  index  are  included.  (TJK) 
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16-34  Numbers  of  students  assessed  in  civics  bridge,  age  D  and  age  17  383 

16-35  Weighted  percentage  of  students  in  main  sample  by  subgroup,  grade  4/age  9  384 

16-36  Weighted  percentage  of  students  in  main  sample  by  sul^oup,  grade  8/age  13  385 

16-37  Weighted  pc  ccntage  of  students  in  main  sample  by  sul^oup,  grade  12/age  17  386 

16-38  Weighted  percentage  of  students  in  bridge  to  19S4  sample  by  subgroup,  grade  4/age  9  387 

16-39  Weighted  percentage  of  students  in  bridge  to  1984  sample  by  subgroup,  grade  8/age  13  388 

16-40  Weighted  percentage  of  students  in  bridge  to  1984  sample  by  sut^oup,  grade  11/agc  389 

16-41  Weighted  percentage  of  students  in  bridge  to  1986  sample  by  sul^oup,  age  9  390 

16-42  Weighted  percentage  of  students  in  bridge  to  1986  sample  by  subgroup,  age  13  391 

16-43  Weighted  percentage  of  students  m  bridge  to  1986  sample  by  suogroup,  grade  11/agc  17  392 

16-44  Weighted  percentage  of  students  in  ci'/ics  bridge  by  suh^oup,  age  13  393 

16-45  Weighted  percentage  of  students  in  civics  bridge  by  sut^oup,  age  17  394 

16-46  Weighted  percentage  of  excluded  students  by  subgroup,  grade  4/age  9  main  and  brid,^c 

samples  395 

16-47  Weighted  percentage  of  excluded  students  by  subgroup,  grade  8/age  13  main  and  bridge 

samples  396 

16-48  Weighted  percentage  of  excluded  students  by  subgroup,  grade  12/age  1?  main  sample  397 

16-49  Weighted  percentage  of  excluded  students  by  sul^oup,  grade  11/agc  17  bridge  sample  398 

16-50  Numbers  of  students  in  main  sample  vnth  proficiency  scores,  grade  4/age  9  399 

16-51  Numbers  of  students  in  main  sample  vnth  proficiency  scores,  grade  8/age  13  400 

16-52  Numbers  of  students  in  main  sample  vnth  proficiency  fcores,  gradr  12/age  17  401 

16-53  Numbers  of  students  in  bridge  tc  1984  sample  vwth  p.ofidency  scores,  grade  4/age  9  402 

16-54  Numbers  of  students  in  bridge  to  1984  sample  vnth  proficiency  scores,  grade  8/agc  13  403 

16-55  Numbers  of  students  In  bridge  to  1984  sample  vnth  proficiency  scoies,  grade  11/age  17  404 

16-56  Numbers  of  students  in  bridge  to  1986  s<.mp'e  vnih  proficiency  scores,  age  9  405 

16-57  Numbers  of  students  in  bridge  to  1986  sa*<iple  vwth  proficiency  scores,  age  13  406 

16-58  Numbers  of  students  in  bridge  to  1986  sample  vnth  proficiency  scores,  grade  11/age  17  407 

16-59  Numbers  of  stadeats  ir*  dvics  bridge  vnth  profidency  scores,  age  13  408 

16-60  Numbers  of  students  m  dvics  bridge  vnth  profidency  scores,  age  i7  409 

16-61  Weighted  profidency  neans,  '>tandard  deviations,  and  percentiles  vnth  standard 

errors  for  main  focused-BIB  reading  samples,  by  gr^ide  and  subgroup  410 

16-62  Weighted  profidenc/  Cvins,  standard  deviations,  a7id  percentiles  vAih  standard 

errors  for  bridge  to  1984  reading  samples,  by  age  and  subgroup  4i2 
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Tabic  16-63   Weighted  profidency  ir.eans,  standard  deviations,  and  percentiles  with  standard 

errors  for  main  writing  samples,  by  grade  and  subgroup  414 
16-64   Weighted  proficiency  means,  standard  deviations,  and  percentiles  with  standard 

errors  for  bridge  to  1984  writing  samples,  by  grade  and  subgroup  416 
16-65   Weighted  proficiency  mUui^  standard  deviations,  and  percentiles  with  standard 

errors  for  main  focuscd-BIB  civics  samples,  by  grade  and  subgroup  418 
16-66   Weighted  proficiency  mcaxis,  standard  deviations,  and  percentiles  with  standa-d 

errors  for  civics  bridge  ramples,  by  age  and  subgroup  ^20 
16-67   Weighted  proficiency  means,  standard  deviations,  and  percentiles  with  standard 

errors  for  main  U.S.  history  samples,  by  grade  an  J  sul^oup  422 
16-68   Weighted  proficiency  means,  standard  deviations,  and  pcrcf  ntilcs  u'lth  standard 

errors  for  mam  geography  sample,  by  grade  and  subgroup  424 
16-69   Weighted  proficiency  means,  standard  deviations,  and  pci^ntilcs  with  standard 

errors  for  bridge  to  1986  mathematics  samples,  by  age  and  subgroup  426 
16-70   Weighted  proficiency  means,  standard  deviations,  and  percentiles  with  standard 

errors  for  bridge  to  1986  science  samples,  by  age  and  subgroup  428 
16-71   Weighted  response  percentages  and  reading  proficiency  means  with  standard 

errors,  grade  4,  by  gender,  for  main  focused-BIB  reading  samples  430 
16-72   Weighted  response  percentages  and  reading  proficiency  means  with  standard 

errors,  grade  4,  by  derived  racc/cf'  "zity,  for  main  focused- BIB  reading 

samples  431 
16-73   Weighted  response  percentages  and  reading  proficiency  means  mih  standard 

errors,  grade  4,  by  parental  education,  for  main  focused-BIB  reading  samples  432 
16-74   Weighted  response  percentages  and  reading  proficiency  means  with  standard 

errors,  grade  8,  by  gender,  for  main  focused-BJB  reading  samples  433 
16-75   Weighted  response  percentages  and  reading  profrdency  means  with  standard 

ci.ors,  grade  8,  by  derived  racc/ethrtidty,  for  main  focuscd-BIB  reading 

samples  434 
16-76   Weighted  response  percentages  and  reading  profidcncy  means  with  standard 

errors,  grade  8»  by  parental  education,  for  main  focused-BIB  reading  samples  435 
16-77   Wcighicd  response  percentages  and  reading  profidency  means  with  standard 

errors,  grade  12,  by  gender,  for  main  focuscd-BIB  reading  samples  436 
16-78   Weighted  response  percentages  and  reading  profidency  means  with  standard 

errors,  grade  12,  by  derived  race/ethnidty,  for  main  ^Dcused-BIB  reading 

samples  437 
16-79   Wetted  response  percentages  and  reading  proficiency  means  with  standard 

errors,  grade  12,  by  parental  education,  for  main.  focused-BIB  reading  samples  438 
16-80   Weighted  response  percentages  and  writing  profidency  means  with  standard 

errors,  grade  4,  by  gender,  for  main  writing  samples  439 
16-81   Weighted  response  percentages  and  writing  profidency  means  with  standard 

errors,  grade  by  derived  race/ethnidty,  for  main  writing  samples  440 
16-82  Wcight(ul  response  percentages  and  writing  profidency  means  with  standard 

errors,  grade  4,  by  paicntal  education,  for  mam  writing  samples  441 
16-83   Weighted  response  percentages  and  writing  profidency  means  with  standard 

en/ors,  g;rade  8,  by  gender,  for  main  writing  samples  442 
16-84   Weighted  response  percentages  and  writing  profidency  means  with  standard 

errorc,  gxade  8,  by  derived  race/ethnidty,  for  main  writing  samples  443 
16-85  Weighted  response  percentages  and  writing  profidency  means  w'lh  standard 

errors,  grade  8,  by  parental  education  for  main  writing  samples  444 
16-86   Weighted  response  percentages  and  writing  profidency  means  with  standard 

errors,  grade  12,  by  gender,  for  main  writing  samples  445 
16-87   Weighted  response  percentages  and  writing  profidency  means  with  standard 

errors,  grade  12,  by  derived  race/ethnidty,  for  main  v^fiiting  samples  446 
16  S8   Weighted  response  percentages  and  writing  proficiency  means  wth  stand^rr* 

errors,  grade  12,  by  parental  education,  for  main  writing  samples  447 
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Table  16-89   Weighted  response  percentages  and  dvics  proficiency  means  with  standard 

errors,  grade  4,  by  gender,  for  main  focused-BIB  civics  samples  448 
16-90   Weighted  response  percentages  and  civics  proficiency  means  with  standard 

errors,  grade  4,  by  derived  racc/ethnidty,  for  main  focused-BIB  civics  samples  449 
16-91   Weighted  response  percentages  and  civics  proficiency  means  wih  standard 

errors,  grade  4,  by  parental  education,  for  main  focused-BIB  civics  samples  450 
16-92   V/eighted  response  percf;ntages  and  civics  proficiency  means  wth  standard 

errors,  grade  8,  by  gender,  for  mam  focused-BIB  civics  samples  451 
16-93   Weighted  response  percentages  and  civics  proficiency  means  with  standard 

errors,  grade  8,  by  derived  race/ethnidty,  for  mam  focused-BIB  dvics  samples  452 
16-94   Weighted  response  percentages  and  dvics  profidency  means  mih  standard 

errors,  grade  8,  by  parental  education,  for  main  focused-BIB  dvics  samples  453 
16-95   Weighted  response  percentages  and  dvics  profidency  means  wth  standard 

errors,  grade  12,  by  gender,  for  main  focused-BIB  dvics  samples  454 
16-96   Weighted  response  percentages  and  civics  profidency  means  with  standard 

errors,  grade  12,  by  derived  race/'ethnidty,  for  main  focused-BIB  dvics  samples  455 
16-97   Weighted  response  percentages  and  d"wcs  profidency  means  with  standard 

errors,  grade  12,  by  parental  education,  for  main  focused-BIB  dvics  samples  456 
16-98   Weighted  response  percentages  and  U.S.  history  profidency  means  with 

standard  errors,  grade  4,  by  gender,  for  mam  U^.  history  samples  457 
i:>-99   Weighted  response  percentages  and  U.S.  history  profidency  means  with 

standard  errors,  grade  4,  by  derived  race/ethnidty,  for  main  U.S.  history 

samples  458 
16-100  Wetted  response  percentages  and  U.S.  history  profidency  means  with 

standard  errors,  grade  4,  by  parental  education,  for  mam  history  samples  459 
16-101  Weighted  response  percentages  and  U.S.  history  profidency  means  with 

standard  errors,  grade  8,  by  gender,  for  mam  US.  history  samples  460 
16-102  Weighted  response  percentages  anci  U.S.  history  profidency  means  with 

standard  errors,  grade  8,  by  derived  race/ethnidty,  for  main  U.S.  history 

samples  461 
16-103  Wetted  response  percentages  and  U.S.  history  profidency  means  with 

standard  errors,  grade  8,  by  parental  education,  for  mam  U^.  history  samples  462 
16-104  Weighted  response  percentages  and  U.S.  history  profidency  means  with 

standard  errors,  grade  12,  by  gender,  for  main  U.S.  history  samples  463 
16-105  Weighted  response  peraintages  and  Ui>.  ristory  profidency  means  with 

standard  errors,  grade  12,  by  derived        ?thnidty,  for  main  U.S.  history 

samples  464 
16-106  Weighted  response  percentages  and  l'.S.  history  proficiency  means  wth 

standard  errors,  grade  12,  by  parental  education,  for  msun  U.S.  history  samples  465 
16-107  Weighted  response  percentages  and  geography  profidency  means  with  standard 

errors,  grade  12,  by  gendei,  for  msun  geography  samples  466 
16-108  Weighted  response  percentages  and  geography  profidency  means  with  standard 

errors,  grade  12,  by  derived  race/ethnidty,  for  main  geography  samples  467 
16-109  Weighted  response  percentages  and  geography  profidency  means  with  standard 

errors,  grade  12,  by  p-^rental  education,  for  main  geography  samples  468 
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Focusing  the  New  Design:    The  NAEP  1988  Technical  Report 

INTRODUCTION 


Albert  E.  Beaton 


Educational  Testing  Service 


The  1988  National  Assessment  of  Educational  Progress  (NAEP)  surveyed 
what  students  in  American  schools  knew  and  could  do  in  the  subject  areas  of 
reading,  vnriting,  civics,  U.S.  history,  and  geography.    Small-scale  studies 
were  also  carried  out  for  mathematics  and  science.    The  populations  that  were 
sampled  included  students  enrolled  in  both  public  and  private  schools  at  ages 
9,  13,  and  17  as  vjell  as  students  in  grades  4,  8,  and  12.     Geography  was 
surveyed  at  grade  12/age  17  only. 

The  purpose  of  this  technical  report  is  to  supply  the  details  of  the 
design  and  data  analysis  of  the  1988  assessment.    Our  aim  is  to  give  the 
reader  sufficient  information  to  judge  the  utility  of  the  design,  the  quality 
of  the  NAEP  data,  the  reasonable* less  of  the  assumptions  made,  the 
appropriateness  of  the  data  analyses,  and  the  general izability  of  the 
inferences  made  from  the  data.    For  educational  psychometricians  and 
statisticians,  the  report  provides  a  summary  of  how  the  technical  challenges 
posed  by  the  1988  assessment  were  addressed  by  NAEP  staff.    For  test 
development  experts,  the  report  provides  a  detailed  account  of  the  evolution 
and  final  composition  of  the  1988  assessment  instruments. 

The  report  does  not  attempt  to  provide  substantive  results  that  might  be 
of  interest  to  educational  policy  makers;  results  from  the  1988  assessment  are 
provJ Jfcd  in  a  series  of  NAEP  reports  on  the  status  of  and  trends  in  student 
performance^.    This  technical  documentation  is  intended  to  support  the 


^The  Reading  Report  Card,  1971  to  1988:    Trends  from  the  Nation's  Report 
Card  (Mullis  &  Jenkins,  1990);  Learning  to  Read  in  Our  Nation's  Schools: 
Instruction  and  Achievement  in  1988       Grades  4,  8,  and  12  (Langer,  Applebee, 
Mullis,  &  Foertsch,  1990);  The  Writing  Rer  :t  Card,  1984-88:    Findings  from 
the  Nation's  Report  Card  (Applebee,  Langer,  Mullis,  &  Jenkins,  1990);  Learning 
to  Write  in  Our  Nation's  Schools:  Instruction  and  Achievement  in  1988  at 
Grades  4,  8,  and  12  (Applebee,  linger,  Mullis,  Jenkins,  &  Foertsch,  1990);  The 
1988  Civics  Report  Card:    Trends  in  Achievement  from  1976  to  1988  at  Ages  13 
and  17,  and  Achievement  in  1988  at  Grades  4,  8,  and  12  (Anderson,  Jenkins, 
Leming,  MacDonald,  Mullis,  Turner,  &  Wooster,  1990);  The  U.S.  History  Report 
Card:    The  Achievement  of  Fourth-,  Eighth-,  and  Twelfth-grade  Students  in  1988 
and  Trends  from  1986  to  1988  in  the  Factual  Knowledge  of  High- school  Juniors 
(Hammack,  Hartoonian,  Howe,  Jenkins,  Levstik,  MacDonald,  Mullis,  &  Owen, 
1990);  and  The  Geography  Learning  of  High-school  Seniors  (Allen,  Bettis, 
Kurfman,  MacDonald,  Mullis,  &  Salter,  1990). 
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proficiency  reports  by  presenting  detailed  information  on  the  methods  used  to 
derive  the  results  that  are  presented  in  those  reports. 

The  technical  details  of  ths  design  and  analysis  of  the  1988  NaEP 
assessment  were  the  result  of  a  collaborative  effort  of  a  large  number  of 
persons,  including  not  only  the  Educational  Testing  Service  (ETS)  and  Westat, 
Inc.,  staff  members  who  collaborated  to  produce  this  report,  but  many  others 
who  contributed  recommendations,  reviews,  comments,  and  other  substantial  help 
on  technical  issues.    Of  special  note  are  the  contributions  of  the  staff  of 
the  National  Center  for  Education  Statistics,  including  Em jrson  Elliott,  David 
Sweet,  Gary  Phillips,  and  Eugene  Owen,  and  of  the  NAEP  Design  and  Analysis 
Committee  (DAG),  chaired  by  Professor  Robert  Linn  (University  of  Colorado). 
Other  members  of  the  committee  included  vice -chair  Professor  Sylvia  Johnson 
(Howard  University) ,  Professor  Emeritus  John  B.  Carroll  (University  uf  North 
Carolina),  Professor  Robert  Glaser  (University  of  Pittsburgh),  Professor  Bert 
Green  (Johns  Hopkins  University),  Professor  Ingram  Olkin  (Stanford 
University),  Dr.  Tej  Pandey  (California  Department  of  Education),  Professor 
Richard  Snow  (Stanford  University),  and  Professor  Emeritus  John  W.  Tukey 
(Princeton  University). 


FEATURES  OF  NAEP  IN  1988 

NAEP  has  always  been  innovative.    When  NAEP  first  collected  assessment 
data  in  1969,  it  introduced  a  number  of  features  that  were  to  lead  the  way  in 
assessment  methodology.    Then,  as  now,  the  National  Assessment  elected  to 
sample  a  broad  range  of  knowledge  and  skills  in  the  subject  areas  that  were 
surveyed,  and  so  introduced  multiple  matrix  sampling  as  a  way  to  enlarge  the 
assessment  coverage  without  placing  excessive  demands  on  the  school  time  of 
individual  students.    NAEP  eschewed  exclusive  reliance  on  multiple-choice 
items,  and  used  many  open-ended  and  essay  items  in  its  assessment  instruments. 
When  funds  permitted,  NAEP  assessments  used  hands-on  methods  of  measuring 
student  performance.    NAEP  applied  and  improved  sampling  methods  and 
procedures  for  estimating  sampling  errors.    A  brief  suimnary  of  innovations  in 
NAEP — at  its  beginning  and  at  this  time — is  available  in  NAEP;    On  the  Cutting 
Edge  of  Measurement  Since  1969  (ETS,  1990). 

The  NAEP  design  underwent  a  series  of  major  modifications  after  1983, 
when  ETS  became  the  grantee.    The  major  goals  of  NAEF  were  maintained,  but  the 
technology  by  which  they  were  attained  was  changed.    The  new  design  that  was 
introduced  is  described  in  A  New  Design  for  a  New  Era  (Messick,  Beaton,  & 
Lord,  1983).    One  feature  of  the  new  design  was  the  introduction  of  modern  IRT 
(item  response  theory)  scaling  technology  in  order  to  summarize  efficiently 
the  extensive  NAEP  data  and  to  communicate  the  results  more  effectively  to 
educational  policy  makers  and  the  public.    Another  design  innovation  was  the 
introduction  of  a  more  complex  form  of  multiple  matrix  sampling  called  BIB 
(balanced  incomplete  blo^k)  spiraling.    BIB  spiraling  made  it  possible  to 
maintain  the  broad  coverage  of  each  subject  arta  while  adding  the  ability  to 
estimate  the  correlations  among  items,  without  increasing  the  amount  of 
testing  time  for  individual  students.    Another  innovation  was  collecting 
information  from  a  national  probability  sample  of  students  who  were  excluded 
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from  the  assessment  (approximately  5  percent)  becaure  they  had  limited  English 
proficiency,  were  mildly  mentally  retarded  (educable),  or  were  functionally 
disabled.    Another  new  feature  introduced  in  the  li84  NAEP  was  the  extension 
oi'  the  sample  to  cover  the  modal  grades  of  the  sti  dents  at  each  age,  so  that 
results  could  be  reported  by  grade  as  well  as  by  ige.    A  new  feature 
introduced  in  1986  was  oversampling  of  Black  and  Hispanic  students,  so  as  to 
improve  the  precision  of  stati.  ics  for  these  sul populations.     The  new  design 
also  2Xtended  the  amount  of  teacher  and  school  information  that  was  made 
available  for  analysis. 

The  new  design  was  introduced  in  the  1984  assessment  and  used  again  with 
some  slight  changes  in  the  1986  assessment.    The  1988  NAEP  assessment  further 
improved  the  r.eneral  design  and  added  a  few  new  features  that  are  discussed 
below. 

It  is  useful  at  this  point  to  consider  the  consequences  of  introducing  a 
new  design  into  an  existing  measurement  system.    There  is  a  clear  tension 
between  the  need  to  maintain  constant  measurement  procedures  in  order  to 
estimate  changes  in  performance  and  the  desire  to  continue  to  improve  the 
assessment  by  using  the  most  modern,  best  available  technology.    The  new 
design  introduced  in  1984  responded  to  this  tension  by  assessing  student 
achievement  in  two  ways:     in  one  set  of  samples  using  the  methods  of  past 
assessments  and  in  another  set  using  the  best  available  methodology.  The 
samples  using  the  methods  of  the  past  were  called  "bridge"  samples,  since  they 
provided  bridges  to  the  performance  of  students  in  past  assessments.  The 
result  was  parallel  assessments,  using  different  technologies,  that  could  be 
compared  and  for  some  purposes,  perhaps,  equated.     In  this  way,  innovations 
could  bft  introduced  without  losing  comparability  with  the  past.    Although  this 
flexibility  to  introduce  innovations  while  maintaining  trends  has  come  at  the 
cost  of  increased  complexity,  the  flexibility  does  allow  NAEP  to  be  responsive 
to  the  information  needs  of  policy  makers  while  maintaining  the  scientific 
requirements  of  sophisticated  survey  research. 

The  innovative  features  of  NAEP  in  1988  were  as  follows: 

Focused'BIB  spiraling.    BIB  spiraling  is  a  special  type  of  multiple 
matrix  sampling  in  which  each  item  in  an  assessment  is  paired  with  each  other 
item  in  some  booklet  so  that  the  interrelationship  between  any  pair  of  items 
can  be  estimated.    As  originally  implemented  in  the  1984  and  1986  national 
assessments,  th    pool  of  all  assessment  items  in  all  subje':t  areas  was  divided 
into  item  blocks  that  typically  took  a  student  about  15  minutes  to  finish. 
The  item  blocks  were  then  combined  in  such  a  way  that  each  blcck  was  pal  'ed 
with  each  other  item  block  in  some  asse5,sment  booklet.    Many  assessment 
booklets  were  printed  and  "spiraled"  together  in  a  random  sequence. 
Ultimately,  each  student  was  assigned  an  assei^sment  booklet  that  contained  a 
block  of  background  and  attitude  questions  and  three  blocks  of  assessment 
items.    As  a  result  of  this  process,  a  student  might  receive  a  booklet  with 
items  from  different  subject  areas:     In  1984,  reading  and  writing  were  BIB- 
spiraled  together;  in  1986,  reading,  mathematics,  science,  and  computer 
competence  were  BIB-spiraled  together.    The  advar tage  of  BIB  spiraling  across 
different  subject  areas  was  that  the  correlations  among  the  various  subject 
areas  could  be  explored.    The  disadvantage  of  this  type  of  BIB  spiraling  was 
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that  many  different  assessment  booklets  had  to  be  printed  and  assigned  to  a 
©mall  number  of  students.    The  correlations  between  subject  areas  were, 
therefore,  based  on  small-sized  samples. 

Focused-BIB  spiraling  was  introduced  in  1988  to  increase  the  sample  size 
on  which  correlations  within  a  subject  area  were  based,  at  the  cost  of 
eliminating  the  correlations  between  items  in  different  subject  areas.  In 
this  case,  an  assessment  booklet  consists  of  a  block  of  demographic  background 
questions,  a  block  of  questions  about  educational  experiences  in  a  specific 
subject  area,  and  three  blocks  of  assessment  items  all  in  that  same  subject 
area.    Proficiency  for  each  student  is,  therefore,  well  measured,  Lat  onl^  in 
one  subject  area.    Using  this  method,  far  fewer  booklets  need  to  be  formed, 
and  each  different  booklet  is  administered  to  a  larger  sample  of  students. 
The  disadvantage  of  the  focused-BIB  spiral  design  is  that  no  Information  is 
accumulated  about  the  relationship  between  different  subject  areas.    To  allow 
the  estimation  of  correlations  between  some  of  the  subject  areas,  the  1988 
design  also  included  several  special  booklets  that  included  blocks  of  items 
from  different  subject  areas. 


Scaling.    The  1988  assessment  continued  the  scaling  advances  that 
characterized  the  j.^84  and  1986  assessments.    The  data  collected  using  the 
focused-BIB  spiral  design  were  cprefully  checked  for  unusual  properties  or 
differential  Item  functioning  in  different  subpopulations .    Although  focused- 
BIB  spiraling  assures  that  each  student  is  presented  enough  items,  there  is  no 
way  to  assure  that  the  student  will  answer  enough  items  in  a  subject  area  for 
precise  estimation  of  his  or  her  proficiency.    To  avoid  the  statistical  bias 
introduced  by  having  oubsamples  that  are  inadequately  measured,  the  method  of 
plausible  values  was  used  In  the  data  analyses.     This  method  also  allows  for 
scaling  and  reporting  in  narrowly  defined  content  areas.     In  addition,  it 
provides  consistent  estimates  of  changes  in  proficiency  over  time,  even  if 
there  are  changes  in  the  number  of  Items  per  student  or  the  average  difficulty 
of  these  Items.    The  scales  were  carefully  anchored  to  enhance  public 
interpretation  of  various  scale  points. 


Comparable  instrument  bridges.    The  1988  design  also  contained  two 
equivalent  samples  of  students  at  each  age  level  that  were  included  to  compare 
the  properties  of  different  assessment  instruments.    The  reading  assessments 
In  1984  and  1986  bot'   used  BIB-spiraled  instruments  but  ths  instruments 
differed  In  a  number  of  seemingly  minor  details.     In  comparing  the  1986 
reading  results  to  the  previously  published  trend  information,  the  1986 
results  seemed  anomalous  and  publication  of  these  results  was  suspended  Ui:til 
further  corroborative  evidence  could  be  gathered.    In  1988,  to  further 
investigate  the  reading  anomaly,  two  equivalent  samples  of  5:tudents  were 
selected  at  each  age  level,  one  of  which  was  assigned  a  booklet  from  the  1984 
assessment  and  the  other  of  which  was  assigned  a  booklet  from  the  1986 
assessment.     (Because  the  booklets  from  the  1986  assessment  contained 
njathematics  and  science  items,  data  for  these  two  subject  areas  were  also 
collected  and  analyzed.)    In  both  cases,  the  administrative  procedures  of  the 
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corresponding  past  assessments  were  duplicated  as  carefully  as  possible. 
Since  the  two  1988  samples  were  randomly  equivalent,  they  are  in  principle 
identical  in  reading  proficiency,  and  any  difference  between  them  is  due  to 
differences  in  the  assessment  instruments  and  sampling  error.    These  samples 
were  subjected  to  extensive  analysis,  and  the  results  were  published  in  a 
separate  report,  entitled  The  Effect  of  Changes  in  the  National  Assessment: 
Disentangling  the  NAEP  1985-86  Reading  Anomaly  (Beaton  &  Zwick,  1990). 

Age  cohorts.     In  the  original  design  of  NAEP  and  in  the  1984 
implementation  of  the  new  design,  the  way  in  which  age  was  defined  at  age  17 
was  different  from  the  way  it  was  defined  at  ages  9  and  13.    As  a  result  of 
these  age  detlttitlons ,  although  most  9-year-olds  were  in  tl  a  fourth  grade  and 
most  13-year-clds  were  in  the  eighth  grade,  most  of  the  17-y  ,ar-olds  were  in 
the  eleventh  grade — three  instead  of  four  grades  ahead.     Comparisons  across 
ages  were  also  complicated  by  the  previous  NAEP  practice  of  testing  13 -year - 
olds  in  the  fall,  9-yeai-olds  in  the  winter,  and  17-year-olds  in  the  spring  of 
the  assessment  year. 

To  facilitate  comparisons  between  different  age  and  grade  groups,  the 
1988  design  Introduced  new  age  cohorts  in  the  main  part  of  the  NAEP  design. 
The  definicioii       17 -year -old  students  was  changed  so  that  the  definitions  of 
the  ages  are  sisilar  for  the  three  age  groups  and  the  NAEP  9-,  13-,  and  17- 
year-old  student  cohorts  are  precisely  four  years  apart.    The  new  definitions 
now  place  most  of  the  17-year-old  students  in  the  twelfth  grade.     In  addition- 
today's  population  of  9-year-olds  will  be  the  popalation  of  13-year-olds 
sampled  in  four  years  and  the  population  of  17-year-olds  sampled  in  eight 
years.    In  these  samples,  students  at  all  ages  and  grades  are  assessed  in  two 
randomly  equivalent  half- samples,  one  assessed  between  January  and  mid-March 
and  the  other  between  mid-March  and  Hay. 

This  innovation  in  the  main  NAEP  sample  could  not  be  introduced  for  the 
bridge  samples  used  for  estimating  trends  (since  the  bridge  samples  had  to  be 
comparable  to  samples  from  previous  assessments),  and  thus  the  bridge  samples 
have  maintained  the  former  age  definitions  and  times  of  year  for  testing. 


Experimental  samples.    Over  the  years,  NAEP  has  developed  a  highly 
successful  and  innovative  way  of  devijloping  assessment  items.    However,  it  is 
continually  exploring  new  ways  to  impiTove  its  procedures.     In  1988,  two 
additional  samples  were  added  to  NAEP  to  explore  potential  measurement 
improvements.    One  sample  involved  the  assessment  of  writing — students*  were 
given  twice  as  much  time  to  write  essays  as  in  the  regular  NAEP  assessment, 
and  the  results  of  the  essays  written  under  different  time  conditions  were 
compared.    This  study  is  reported  in  Leaxriing  to  Write  in  Our  Nation's 
Schools:    Instruction  and  Achievement  in  1988  at  Grades  4,  8,  qnd  12 
(Applebee,  Langer,  Mullis,  Jenkins,  &  Foertsch,  1990).    The  second  special 
sample  was  designed  to  link  student  performance  at  ages  13  and  17  to  FAEP's 
1985  literacy  study  of  young  adults. 
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Poststratification  improvements.    As  discussed  in  section  8.1.4  of 
Chapter  8,  the  process  of  computing  the  poststratif ication  adjustments  was 
modified  from  the  procedures  followed  in  1984  and  1986.    TYie  changes  were 
introduced  to  accelerate  the  reporting  process  and  to  make  NAEP  results  closer 
to  those  of  the  Census  Bureau  in  terms  of  numbers  of  students  reported  in 
various  subpopulations  identified  by  region,  race/ethnicity,  age,  and  grade. 


Teacher  questionnaires.     The  design  for  administering  teacher 
questionnaires  was  modified  in  1988  to  obtain  teacher  data  for  all  students  at 
a  given  grade  in  a  particular  subject  area,  rather  than  for  a  few  students  in 
all  subject  areas.     In  1988,  resources  permitted  obtaining  teacher  data  for 
two  samples  of  students.     Extensive  teacher  questionnaires  were  given:     1)  to 
teachers  of  fourth- grade  students  who  were  assessed  in  reading  and  2)  to 
teachers  of  eighth-grade  students  who  were  assessed  in  writing.  These 
questionnaires  contain  not  only  questions  about  the  teacher's  background  and 
teaching  practices  but  also  questions  for  the  teacher  about  the  performance  of 
his  cr  her  individual  students.    The  results  of  the  assessment  of  student 
performance  and  the  teacher  questionnaires  have  been  linked  for  analysis  and 
reported  in  Learning  to  Read  in  Our  Nation's  Schools:    Instruction  and 
Achievement  in  1988  at  Grades  4,  8,  and  12  (Langer,  Applebee,  Mullis,  & 
Foertsch,  1990)  and  Learning  to  Write  in  Our  Nation's  Schools:  Instruction  and 
Achievement  in  1988  at  Grades  4.  8,  and  12  (Applebee,  Langer,  Mullis,  Jenkins, 
&  Foertsch,  1990). 


Public-use  data  tapes  and  other  database  products.    NAEP  continues  to 
make  available  to  the  public  all  of  its  data,  except  those  that  would  identify 
its  participants.    The  public-use  data  tapes  are  carefully  prepared  and 
documented,  making  it  possible  for  others  to  duplicate  the  analyses  done  by 
NAEP  staff  or  conduct  other  analyses  of  the  vast  NAEP  database. 

The  NAEP  item  information  database  concains  ell  of  the  des crip rive, 
processing,  and  usage  information  for  every  item  developed  and  administered 
for  NAEP.     This  database  functions  as  a  resource  for  test  development 
activities,  data  system  control  operations,  and  item  linkage  to  past 
assessments . 

The  NAEP  restricted-use  data  files  contain  all  NAEP  respondent  data, 
including  "secure"  files.    These  files  function  as  NAEP  data  archives  for 
responses  from  students,  teachers,  and  school  administrators  from  the  booklets 
and  questionnaires  used  in  NAEP  from  1970  to  1988. 

The  NAEP  database  products  are  especially  useful  because  they  are 
portable  and  can  be  used  on  a  variety  of  hardware  systems;  they  can  be 
accessed  by  a  variety  of  software  systems  (including  SAS  and  SPSS);  they  are 
in  a  "rectangular"  file  structure  that  eliminates  the  need  for  complex  data 
retrieval  processes  from  dissimilar  file  formats;  and  they  are  well 
documented. 
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ORGANIZATIOH  OF  THE  TECHNICAL  REPORT 


This  technical  report  is  divided  into  three  parts: 

Part  I  presents  the  details  of  the  1988  design  and  contains  a  summary  in 
Chapter  1  of  the  steps  involved  in  the  process  of  producing  a  database  ready 
lor  analysis.    Chapter  1  is  followed  by  chapters  in  which  the  details  of  the 
snveral  steps  in  collecting  and  preparing  the  data  for  analysis  are  described. 
Chapter  2  describes  the  specification  of  the  NAEP  assessment  objectives  and 
the  development  of  the  assessment  items  and  background  questions  for  the 
several  assessment  instruments  and  questionnaires.    Chapter  3  describes  the 
details  of  the  selection  of  the  NAEP  samples.    The  details  of  the  a^sassment 
instruments  that  were  used  in  the  1988  NAEP  are  presented  in  Chapter  4.  The 
adrainirtration  of  "ASP  to  ctudents  in  American  schools  is  discusse.   in  Chapter 
5.    Chapters  6  through  6.7  contain  the  details  of  converting  the  data  received 
from  the  field  into  a  usable  database,  including  the  processing  of  the  raw 
assessment  materials,  professional  scoring,  data  transcription  systems, 
editing,  quality  control,  and  the  creation  of  the  database  system,  and 
desL^ibes  the  database  products  that  are  available. 

Part  II  begins  with  Chapter  7,  which  outlines  the  analysis  of  the  1988 
NAEP  data.    Chapter  8  includes  general  discussions  of  the  weighting  and 
variance  estimation  procedures  use*^  *n  NAEP  analyses.    A  general  discussion  gf 
the  NAEP  scaling  methodology  is  pre    nted  in  Chapter  9.    Chapters  10  through 
15  contain  the  details  of  the  analyses  performed  for  the  respective  subject 
areas . 

Part  III  contains  some  bas^c  data  from  the  1988  NAEP  assessment, 
including  the  properties  of  th^  measuring  instruments,  characteristics  of  the 
selected  sample,  and  some  estimates  of  the  proficiencies  of  students  in 
American  schools.    Only  a  few  of  the  huge  number  of  '"-^ssible  population 
proficiency  estimates  are  presented,  and  these  include  estimates  of  the  means, 
standard  deviations,  and  selected  percentiles  of  populations  of  students  in 
various  su>ject  areas  and  grade  levels.     Estimates  are  also  presented 
separately  for  gender,  racial/ethnic  groupings,  and  other  subpopulations . 
Estimates  of  average  proficiency  values  for  cross-classifications  of  selected 
variables  are  also  reported.    Estimated  standard  errors  are  reported  with  all 
parameter  estimates. 
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PART  J 

The  Design  and  Implementation  of  the  1988  NAEP 
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Chapter  1 


OVERVIEVf  OP  PART  I:    THE  DESIGN  AND  IMPLEMENTATION 
OP  THE  1988  NAEP^ 


Eugene  G.  Johnson 
Educational  Testing  Service 


The  1988  National  Assessment  collected  infonuation  on  rhe  knowledge, 
skills,  understanding,  and  attitudes  of  young  Americans  in  the  subject  areas 
of  reading,  writing,  civics,  U.S.  histoiry,  and  geography.    In  addition, 
information  on  mathematics  and  science  was  collected  in  a  special  small-scale 
study.    The  basis  for  this  information  was  a  complex  sample  survey  involving 
more  than  126.000  students  and  consisting  of  national  sam^^Bs  of  students  aged 
9,  13,  and  17       vel]  as  national  samples  of  students  in  grades  4,  8,  11,  and 
12.    This  chapter  p3ro^^ides  a  dtsscription  of  the  design  for  the  1988  assessment 
and  gives  an  overview  of  the  steps  involved  in  the  implementation  of  NAEP  from 
the  planning  stage  through  the  creation  of  a  database  ready  for  analysis.  The 
major  components  of  the  implementation  of  the  assessment  are  presented  here 
with  references  to  the  appropriate  chapters  in  Part  I  for  more  det^^ils.  Not 
included  in  Part  I  are  the  procedures  used  for  the  analysis  of  the  data,  these 
are  summarized  in  the  overview  to  Tdict  II  (Chapter  7)  and  discussed  In  detail 
in  the  remaining  chapters  in  Part  H. 

The  organization  of  this  chapter,  and  of  Part  I,  is  as  follows: 

•  Section  1.1  provides  an  overview  of  the  NAEP  design  for  1988  and 
describes  the  constituent  samples.    To  provide  background,  the 
section  also  gives  the  assessment  schedule  from  the  inception  of 
NAEP  in  1969  through  the  1988  assessment. 

•  Section  1.2  summarizes  the  four- stage  stratified  random  sampling 
procedures  used  for  the  1988  assessment  with  a  fuller  description 
provided  in  Chapter  3. 

•  Section  1.3  summarizes  the  development  of  the  objectives  for  each 
of  the  subject  areas  in  the  assessment  ind  the  development  and 
review  of  the  items  written  to  fit  those  objectives.    Details  of 
the  objective  and  item  development  processes  appear  in  Chapter  2. 

•  Section  1.4  discusses  the  ai^signment  of  the  cognitive  and 
background  questions  to  assessment  booklets  and  describes  the 


^The  author  is  indebted  to  Albert  Beaton  and  the  authors  of  Chapters  2 
through  6  for  portions  of  this  chapter,  and  to  Mary  Varone  for  ably  typing  the 
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focused-BIB  spiral  design.  Chapter  4  provides  a  complete 
description  of  the  assessment  booklets. 


•  Section  1.5  summarizes  the  field  administration  procedures 
including  the  training  of  field  administrators,  attaining  school 
cooperation,  administering  the  assessment,  and  conducting  quality 
control.    Further  details  appear  ir.  Chapter  5. 

•  Section  1.6  describes  the  flow  of  the  data  from  their  receipt  at 
ETS  through  data  entry,  professional  scoring,  and  entry  into  the 
database  in  final  form,  ready  for  analysis.    Chapters  6  through 
6.7  provide  a  detailed  description  of  the  process. 

The  data  collected  in  the  1988  assessment  are  available  for  public  use 
in  a  set  of  data  tapes  containing  the  data  from  the  sampled  students,  data 
about  students  excluded  from  the  assessment,  and  data  from  teachers, 
principals,  and  schools  linked  to  the  assessed  students.    The  documentation 
for  the  public-use  data  tapes  appears  in  National  Assessment  of  Educational 
Progress  198^  Public-use  Data  Tapes  Version  2.0  User  Guide  (Rogers,  Kline, 
Johnson,  Mislevy,  &  Rust,  1990). 

1.1    THE  1988  NAEP  DESIGN 

The  1988  NAEP  design  was  intended  to  address  two  occasionally  competing 
considerations.    The  first  is  NAEP's  charge  to  measure  trends  in  educational 
achievement.    The  second  is  the  need  for  NAEP  to  evolve  as  educational  issues 
of  interest  develop  over  time  and  as  new  technologies  of  assessment  become 
available.    The  goal  of  measuring  trends  requires  a  stability  in  the 
measurement  process;  the  goal  that  NAEP  evolve  implies,  however,  that  the 
measurement  process  must  be  permitted  to  change. 

The  1988  design  addresses  the  competition  between  stability  and  change 
by  basing  the  assessment  on  two  distinct  tjrpes  of  samples.    The  first  type  of 
sample,  the  trend  sample,  is  used  for  estimating  changes  in  performance  from 
previous  assessments  in  a  subject  area  and  uses  the  same  methodology  and 
population  definitions  as  in  previous  assessments.     The  second  type  of  sample, 
a  cross- sectional  sample,  is  used  for  detailed  information  about  the  current 
student  population.    This  sample  allows  the  use  of  new  technology  and 
population  definitions  and  addresses  new  educational  issues. 

A  number  of  improvements  have  been  made  in  the  design  of  NAEP  since  ETS' 
first  assessment  in  1984.    Before  the  1984  assessment,  NAEP  used  a  simple 
matrix  sampling  procedure  with  audiotape  pacing — all  students  in  an  assessment" 
session  received  the  same  booklet  of  assessment  items  and  an  aurally  presented 
stimuljs  was  used  to  pace  the  students  through  the  assessment  items.     In  the 
1984  assessment,  balanced  incomplete  block  (BIB)  spiraling  (discussed  in 
section  1.4),  which  does  not  include  aural  pacing,  was  instituted  in  place  of 
taped  matrix  sampling.    With  BIB  spiraling,  students  in  an  assessment  session 
receive  different  booklets  resulting  in  a  more  efficient  sample  (for  reasons 
given  in  section  1.4).    BIB  spiraling  also  allows  the  study  of  the 

14 

ERiC  ^7 


interrelationships  between  all  items  included  in  the  balanced  incomplete  block 
design.    In  the  1988  assessment,  additional  efficiencies  were  introduced  when 
focused-BIB  spiraling  was  instituted.    Focused-BIB  spiraling  ensures  that  all 
correlations  between  items  within  a  subject  area  can  be  estimated  but,  unlike 
the  BIB  designs  used  in  1984  and  1986,  does  not  require  that  correlations 
between  items  in  different  subject  areas  be  estimable.    This  produces  a 
simpler  and  more  efficient  design  than  that  used  in  1984  and  1986.     Like  BIB 
spiraling,  focused-BIB  spiraling  required  the  elimination  of  the  audiotape 
pacing  used  in  earlier  assessments. 

From  its  inception,  NAEP  has  sampled  students  of  a  given  age  (9,  13,  and 
17).     Since  1984,  the  designs  have  also  included  samples  from  the  population 
of  the  corresponding  modal  grades  (the  grade  in  school  of  the  majority  of  the 
students  of  the  age  level).    As  was  the  case  for  previous  national 
assessments,  the  primary  populations  of  inference  for  the  1988  assessment  were 
in-school  students  of  the  specified  ages  or  grades.     Thus,  youth  of  the 
specified  age  who  were  not  attending  public  or  private  school  at  the  time  of 
the  assessment  (including,  in  particular,  dropouts  and  early  graduates)  were 
excluded  from  the  sample  and  from  the  population  of  inference. 

To  assure  that  the  grade/age  samp  .  .  measure  four  years  of  growth,  the 
definitions  of  student  age  were  made  unif >rm  for  the  1988  assessment.  While 
NAEP  ori  finally  def i-  -d  age  by  birth  within  a  calendar  year  for  ages  9  and  13 
but  defined  age  17  as  being  born  between  October  1  of  one  year  and  September 
30  of  the  next,  the  1988  assessment  placed  all  ages  on  the  calendar-year 
basis.    This  change  in  age  definition  changed  the  modal  grade  for  17-year-old 
students  from  the  eleventh  grade  to  the  twelfth  grade.     Since  their  age 
definitions  were  unchanged,  the  modal  grades  for  ages  9  and  13  remained  at 
grades  4  and  8. 

A  final  change  in  the  1988  assessme.it,  relative  to  previous  assessments, 
in  the  direction  of  greater  uniformity  and,  therefore,  greater  comparability, 
was  in  the  time  of  year  that  the  students  were  assessed.    NAEP  tr^«ditionally 
assessed  9-year-olds  in  the  winter,  13-year-olds  in  the  fall,  and  17-year-olds 
in  the  spring;  in  the  1988  assessment,  all  students  were  assessed  in  the 
winter  and  spring. 

1.1.1    The  1988  Samples 

The  full  1988  assessment  consists  of  four  types  of  samples:  main 
assessment  focused-BIB  samples,  main  assessment  intercorrelation  samples,  main 
assessment  special  study  samples,  and  bridge  (trend)  samples.     A  list  of  all 
assessment  samples,  with  '  ey  characteristics,  appears  in  Table  1-1.  A 
description  of  the  samples  follows. 

Main  assessment  f-        .-BIB  samples.    These  samples  form  the  basis  for 
the  cross -sectional  of  achievement  for  the  1988  student  population 

and  assessed  the  su  ^'eas  of  writing  (labeled  as  [Main-Wrt]  in  Table 

1-1),  reading  [(Mai.  AdgJ ,  U.S.  history  [Main-His] ,  civics  [Main-Civ]  and 
geography  [Main-Geo].    All  but  geography  were  assessed  at  all  three  age 
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Table  1-1 
NAEP  1988  Student  Samples 
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classes;  geography  was  assessed  only  at  grade  12/age  17.     In  these  samples, 
focused-BIB  spiraling  (and  hence  printed  administration)  was  used,  age  was 
defined  on  a  calendar-year  basis  and  both  age  populations  (9,  13,  17)  and 
modal  grade  populations  (4,  8,  12)  were  sampled.     Each  age  class  sample  was 
divided  into  two  random  hr\f -samples ,  one  of  which  was  assessed  in  the  winter 
and  the  other  in  the  spring.    The  purposei>  of  these  half -samples  were:   (1)  to 
allow  comparison  with  other  selected  samples  (assessed  only  in  the  winter  or 
spring) ;  and  (2)  to  allow  the  study  of  growth  in  student  achievement  within  a 
school  year. 

Main  assessment  intercorr elation  samples.    These  samples  (labeled 
[Main-Int]  in  Table  1-1)  are  intended  to  permit  the  measurement  of 
interrelationships  in  achievement  between  subject  areas.     Each  booklet  in 
these  samples  includes  blocks  v^f  reading,  r.ivics,  and  U.S.  history  items  (as 
well  as  some  geography  items  at  grade  12/age  17).     Print  administration  was 
used  and  age  was  defined  on  a  calendar-year  basis.     Both  grades  and  ages  were 
sampled  and  the  assessment  was  conducted  in  two  random  half* samples  in  che 
winter  and  spring. 

Main  assessment  special  study  samples.     These  samples  are  designed  to 
allow  the  conduct  of  special  studies  relating  to  achievement.    The  long 
writing  sample  [Main-LWr]  is  intended  to  measciie  the  relationship  between  time 
allocated  to  the  writing  task  and  writing  performance;  the  document  literacy 
sample  [Main-Doc]  allows  the  assessment  of  document  literacy  and  its  relation 
to  reading  proficiency.    Both  samples  used  printed  administration,  age  and 
grade  sampling,  and  new  age  definitions  and  time  of  testing. 


The  1988  assessment  also  includes  a  number  of  additional  samples 
designed  to  determine  the  possible  effects  of  changes  in  age  definitions,  time 
of  testing,  and  mode  of  administration  (audiotape  pacing  versus  print 
administration)  and  to  provide  links  to  the  results  from  previous  assessments. 
Because  the  purpose  of  these  samples  is  to  provide  a  linkage  between  the  1988 
data  and  data  from  previous  assessments,  they  are  referred  to  as  bridge 
samples  (although  they  are  also  called  trend  samples).    The  vaiious  bridge 
samples  are  as  follows: 

Civics  Bridge  to  1976  and  1982.    This  bridge  (trend)  sanple,  labeled 
[BrCiv]  in  Table  1-1,  addresses  the  subject  area  of  civics.    The  samples  for 
this  bridge  are  comparable  to  past  asse.^sments  of  citizenship  and  social 
studies.    Like  these  past  assessments,  the  civics  bridge  sample  uses  tape 
recorders  and  pre- 1984  definitions  of  age  and  time  of  testing.     Since  trend 
data  have  been  traditionally  collected  onl>  by  age,  grade  sampling  was 
unnecessary.    The  civics  bridge  sample  consists  of  one  booklet  for  age  13  and 
one  booklet  for  age  17.     Because  there  were  no  reusable  civics  items  from 
previous  assessments  of  9 -year- olds,  an  age  9  sample  was  not  needed. 
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Bridge  to  19S4.  This  bridge  (trend)  sample,  labeled  [Br84-RW]  in  Table 
1-1,  consists  of  samples  comparable  to  the  1984  main  assessment  and  addresses 
the  subject  areas  of  reading  and  writing.  The  samples  were  collecteu  by  grade 
and  age  for  grade  4/age  9,  grade  8/age  13,  and  grade  11/age  17,  using  the  age 
definitions  and  time  of  testing  from  1984.  Six  assessment  booklets  were 
administered  at  each  grade/age.  Each  booklet  consisted  of  at  least  one  block 
of  reading  items  and  at  least  one  block  of  writing  items.  The  administration 
of  these  booklets  was  nonpaced  (that  is,  no  audiotape  was  used). 

Bridge  to  1986,  Ages  9  and  13.    This  bridge  consists  of  trend  samples 
for  ages  9  and  13  comparable  to  those  used  for  the  measurement  of  trends  in 
1986.     The  samples  are  labeled  9[Br86-RMS]  and  13[Br86-RMS]  in  Table  1-1.  The 
samples  were  collected  by  age  only  and  used  the  same  age  definitions  and  time 
of  testing  as  in  1984  and  in  the  1986  bridgi*  to  1984.    The  sujject  areas 
addressed  by  this  bridge  are  read    g,  rriathemv-itics,  and  science.  Three 
assessment  booklets  were  administered  to  each  age  group.     These  booklets  were 
identical  to  those  administered  in  1986.    Each  booklet  contains  one  block  of 
reading,  one  block  of  mathematics,  and  one  block  of  science  items.    As  in 
^986,  administration  of  the  mathematics  and  science  blocks  was  paced  with  an 
audiotape;  the  reading  blocks  were  administered  without  an  audiotape. 

Bridges  to  1986,  Grade  11/Age  17.    These  '>ric\,es  (labeled  17[Br86-RMS] 
and  17 [Br86-His] )  consist  of  trend  samples  of  grade  11/age  17  students 
comparable  to  the  1986  main  assessment  sample  and  were  selected  and 
administered  using  the  same  age  definition  and  time  of  uesting  as  in  that 
assessment.     Since  those  definitions  also  apply  to  samples  from  the  1984  and 
earlier  assessments,  the  students  in  these  bridge  samples  are  comparable  to 
the  students  from  these  earlier  assessments.     (However,  the  performance 
results  are  not  directly  comparable  because  the  earlier  assessments  had  paced 
audiotape  administrations.)    The  subject  areas  assessed  in  the  17 [Br86-EIMS] 
bridge  were  reading,  mathematics,  and  science  and  consisted  of  six  assessment 
booklets  administered  to  grade  11/age  17  students.     These  booklets  contained 
blocks  of  reading,  mathematics,  and  science  items.    Tne  17[Br86-His]  bridge 
involved  one  booklet  consisting  entirely  of  blocks  of  U.S.  history  items.  The 
administration  of  the  booklets  in  both  bridges  was  nonpaced. 

Although  many  of  the  questions  'n  the  assessment  booklets  for  the  brld^;e 
samples  also  appeared  in  the  booklets  used  for  the  main  assessment,   the  bridg.^ 
assessment  instruments  were  considered  as  dii>tinct  from  the  main  assessment 
instruments.    Additionally,  the  procedures  used  to  administer  the  bridge 
assessment  instriments  sometimes  differed  from  those  used  for  the  main 
assessment.    A  overview  of  the  assessment  instrumentation  and  mode  of 
administratio*^.  :;ppears  in  section  1.4,  with  further  details  presented  in 
Chapter  4.     Details  of  the  analysis  of  the  bridge  and  main  assessment  data 
appears  in  Part  XI  of  this  technical  report. 
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1.1«2    NAEP  Assessments  Since  U  69 

Table  1-2  shows  the  subject  areas,  grades,  and  ages  assessed  since  the 
inception  of  NAEP  in  1969.    As  can  be  seen,  besides  the  1988  subject  areas  of 
reading,  writing,  civics,  U.S.  history,  geography,  mathematics,  and  science, 
many  other  subject  areas  have  been  assessed  over  the  years — social  studies, 
citizenship,  literature,  music,  career  development,  art,  and  computer 
competence.    Many  subject  areas  have  been  reassessed  periodically  to  determine 
trends  over  time. 

Assessments  yere  conducted  annually  through  1980,  but  budget 
restrictions  since  then  have  reduced  data  collection  to  a  biennial  basis. 
Since  its  inception,  NAEP  has  assessed  9 -year-olds,  13 -year- olds ,  and  in- 
school  17-year-olds,  although  the  age  definitions  changed  in  1986  and  again  in 
1988.    Because  of  budget  restrictions,  NAEP  no  longer  routinely  assesses  out- 
of -school  17-year-olds  or  young  adults.     (A  separate  assessment  of  young 
adults  or  ages  21  to  25  was  conducted  in  1985  under  a  separate  grant.) 

The  table  also  indicates  the  initiation  of  data  collection  by  grade  as 
well  as  by  age  in  1984,  a  practice  that  was  continued  in  the  1986  and  1988 
assessments.     It  should  be  noted  that  somewhat  different  age  definitions  were 
used  in  the  1984,  1986,  and  1988  assessments.     In  the  1984  assessment,  the 
younger  two  ages  were  defined  on  a  calendar-year  basis  while  the  17-year-olds 
were  defined  on  an  October  1  to  September  30  basis.    This  resulted  in  modal 
grades  of  4,  8,  anJ  11.     In  the  1986  main  assessment,  all  ages  were  defined  on 
a  October  1  to  September  30  basis  resulting  in  modal  grades  of  3,  7,  and  11. 
In  1988,  the  ages  were  redefined  on  a  calendar-year  basis,  with  the  modal 
grades  being  4,  8,  and  12. 

1.2    THE  1988  SAMPLE  DESIGN 

The  target  population  for  the  1988  main  assessment  consists  of  all 
students  in  public  and  private  schools  wl  o  belong  to  one  of  three  cohorts: 
students  who  were  either  in  the  fourth  grade  or  9  years  old;  students  who  were 
either  in  the  eighth  grade  or  13  years  old;  and  students  who  were  either  in 
the  twelfth  grade  or  17  years  old.    The  main  assessment  represents  two 
overlapping  samples.    The  first  sample  represents  students  of  the  grades  4,  £, 
and  12  (who  could  be  of  any  age) — these  are  the  modal  grades  for  the  students 
of  the  specified  ages.     The  second  sample  represents  students  of  specified 
ages  (who  could  be  of  any  grade).     Students  were  age-eligible  for  the  main 
assessment  if  th^.y  were  born  in  the  appropriate  calendar  year  (1978,  1974,  or 
1970)*    Only  students  who  were  attending  public  or  private  schools  at  the  time 
of  the  assessment  were  included  in  the  sample  (and,  therefore,  in  the  target 
population).     Specifically  excluded  from  the  sample  and  the  target  population 
are  early  graduates  and  dropouts. 

For  the  purposes  of  analysis,  the  grade/age  samples  were  treated  as  two 
separate  sajnples:  1)  a  representative  sample  of  students  in  grades  4,  8,  and 
12,  and  2)  a  representative  sample  of  students  of  ages  9,  13,  and  17.  (A 
student  who  was  both  grade  and  age  eligible,  was  regarded  as  a  member  of  both 
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Table  1-2 


National  Assessment  of  Educational  Progress 
Subject  Areas,  Grades,  and  Ages  Assessed:  1969-1988 


Assessment 
1969-70 


1970-  71 

1971-  72 

1972-  73 

1973-  74 

1974-  75 

1975-  76 


Subject  Area(s) 


Grades/A^ges  Assessed 
Grade    Grade    Age    Grade    Grade    Age    Grade    Grade     Age  Age 

1        1  SllillliZ    iZOS«  Adult 


Science 
Writing 
Citizenship 

Reading 
Literature 

Music 

Social  Studies 

Science 
Mathematics 

Career  and  Occupational 

Development 
Writing 

Reading 
Art 

Index  of  Basic  Skills 

Citizenship/Social  Studies 
Mathematics^ 


X 
X 
X 

X 
X 

X 
X 

X 
X 


X 
X 

X 
X 


X 
X 
X 

X 
X 

X 
X 

X 
X 


X 
X 

X 
X 


X 
X 


As«  17  itudtnts  who  had  dropped  out  of  school  or  had  sraduatad  prior  to  assessmant. 

Small,  apacial-intarast  assassmant  conductad  on  limitad  saaplas  at  spacific  j^radas  or  a^as. 


X 
X 
X 

X 
X 

X 
X 

X 
X 


X 
X 

X 
X 
X 

X 
X 


X 
X 
X 

X 
X 

X 
X 

X 
X 


X 
X 

X 
X 
X 

X 
X 


X 
X 
X 

X 
X 

X 
X 

X 
X 
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Table  1-2  (continued) 


Assessment 
Year 

1976«77 


National  Assessment  of  Educational  Progress 
Subject  Areas,  Grades,  and  Ages  Assessed:  1969-1988 


Subject  Areafs) 
Science 

Basic  Life  Skills*' 
Science,  Reading,  Health*' 


Grades/Ages  Assessed 
Grade    Grade    Age    Grade    Grade    Age    Grade    Grade     Age  Age 

1  {ill  iililliiZ      170S^  Adult 


X 
X 


1977-  78 

1978-  79 

1979-  80^ 

1981-82 

1984*^ 
1985^ 


Mathematics 
Consumer  Skills** 

Writing,  Art,  and  Music 

Reading/Literature 
Art 

Science^ 

Math  and  Citizenship/ 
Social  Studi  s 


Reading 
Writing 

Adult  Literacy** 


X 
X 


X 
X 


X 
X 


X 
X 


X 
X 


X 
X 


X 
X 


X 
X 


X 
X 


X 
X 

X 

X 

X 

X 

X 
X 


^  As«  17  students  who  had  dropped  out  of  school  or  had  sraduatcd  prior  to  assessment. 

Small,  special-interest  assessment  conducted  on  limitwd  samples  at  specific  grades  or  ases. 
Assessment  conducted  by  Educational  Testins  Service. 


Because  of  reduced  fundinst  assessments  were  conducted  biennially  after  1979-60. 
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Table  1-2  (continued) 


Assessment 
1986^ 


1988* 


National  Assessment  of  Educational  Progress 
Subject  Areas,  Grades,  and  Ages  Assessed:  1969-1988 


Subject  Ar^a(s) 

Reading 

Mathematics 

Science 

Computer  Competence 
U,S,  History^ 
Literature^ 

Reading 
Writing 
Civics 

U,S,  History 
Document  Literacy^ 
Geography^ 
Mathematics** 
Science^ 


Grades /Ag€.s  A?^^e??<^v 
Grade    Grade    Age    Grade    Gxade    Age    Grade    Grade     Age  Age 

^  i22  &       n       11        11        n     170S«  Adult 


X 
X 
X 
X 


X 
X 
X 
X 


X 
X 
X 
X 


X 
X 
X 
X 


X 
X 


X 
X 
X 
X 


X 
X 
X 
X 
X 


X 
X 
X 
X 


X 
X 
X 
X 
X 

X 
X 


X 
X 
X 
X 
X 
X 


X 
X 
X 
X 
X 
X 


X 
X 


X 
X 
X 
X 
X 
X 

X 
X 
X 
X 
X 
X 
X 
X 
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b  ttudtntB  who  had  dropped  out  of  tchool  or  had  sraduatcd  prior  to  asataamant. 

Sm/  I,  apacial-intaraat  aaaaataant  conducted  on  limitad  atoplot  at  apacific  gradaa  or  asas. 
/  ^osmant  conducted  by  Educational  Taatins  Sarvica. 
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of  these  samples.)  Thus,  sununary  statistics  were  computed  tc  the  age  sample 
or  for  the  grade  sample,  but  generally  not  for  the  combined  grade/age  sample. 

The  target  population  for  the  bridge  assessments  also  consisted  of  all 
public  and  private  school  students  in  one  of  three  cohorts,  but  the  age  and 
grade  definitions  for  one  cohort  differed  from  that  used  in  the  main 
assessment.    All  bridge  samples  contained  samples  of  students  eligible  by  age 
using  the  following  age  definitions  (consistent  with  those  used  in  1984  and 
previous  assessments) : 

Students  were  eligible  for  the  age  9  and  the  age  13  bridge  assessments 
if  they  were  bcrn  in  the  appropriate  calendar  year  (1978  and  1974, 
respectively);  students  were  eligible  for  the  age  17  bridge  assessments  if 
they  were  born  between  October  1,  1970  and  September  30,  1971. 

For  certain  bridge  samples,  students  eligible  by  grade  were  also 
selected.     These  samples  consisted  of  students  in  grades  4,  8,  and  11,  the 
modal  grades  for  the  age  definitions  used  by  the  bridge  samples. 

The  sample  for  the  1988  NAEP  assessment  was  selected  using  a  complex 
four- stage  sample  design  involving  the  sampling  of  students  from  selected 
schools  within  94  selected  geographic  areas,  called  primary  sampling  units, 
across  the  United  States.    The  sample  design  is  similar  to  that  used  in  1986 
and  is  described  in  denail  by  Westat,  Inc.,  the  firm  subcontracted  by  ETS  to 
select  the  sample,  in  1988  National  Assessment  of  Educational 
Progress — Sampling  and  Weighting  Procedures ,  Final  Report  (Rust,  Bethel, 
Burke,  &  Hansen,  1990).    The  following  sections  provide  an  overview  of  each  of 
the  four  stages  of  the  sampling  design  with  further  details  given  in 
Chapter  3. 


Stage  1:     Primary  Sampling  Units 

In  the  first  stage  of  sampling,  the  United  States  (the  50  states  and  the 
District  of  Columbia)  was  divided  into  geographic  primary  sampling  units 
(PSUs).    Each  PSU  met  a  minimum  sizt  requirement  and  generally  comprised 
either  a  metropolitan  statistical  area  (MSA),  a  single  county,  or  a  group  of 
contiguous  counties.     The  PSUs  were  classified  into  four  regions:  (Northeast, 
Southeast,  Central,  West),  each  containing  about  one-fourth  of  th<»  U.S. 
population.    In  each  region,  PSUs  were  additionally  classif  '^d  as  MSA  or 
nonHSA.    In  the  Southeast  and  West  regions,  the  PSUs  in  /hicn  20  percent  of 
thtl  population  in  the  1980  Census  was  eicner  Black  or  Hi.' panic  were  furthc 
classified  as  high-minority,  while  the  remaining  PSUs  in  those  regions  weic 
classified  as  not  high-minority.    This  resulted  in  twelve  subuniverses  of 
PSUs. 

\Ninety-four  PSUs  were  then  selected  from  these  subuniverses  with 
probability  proportional  to  a  measure  of  their  size  (the  number  of  school  age 
children  from  the  1980  census).    Thirty-four  large  PSUs  were  designated  as 
certainty  units,  as  it  vas  cost  effective  to  include  them  in  the  sample  with 
certainty.    Within  each  major  stratum  (subuniverse) ,  further  stratification 
was  achieved  by  ordering  the  noncertainty  PSUs  according  to  several  additional 
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socioeconomic  characteristics.    Sixty  PSUs  were  selected  from  the  noncertainty 
strata  with  probability  proportional  to  size.    To  enlarge  the  samples  of  Black 
and  Hispanic  students,  thereby  enhancing  the  reliability  of  estimates  for 
these  groups,  PSUs  from  the  high-minority  subuniverses  werp  sampled  at  twice 
the  rate  of  those  from  the  other  subuniverses. 

The  94  PSUs  were  used  fpr  the  main  assessments  of  all  three  age  classes. 
To  facilitate  and  improve  administration  of  the  assessments  and  to  allow  for 
some  estimation  of  growth  in  achi^evement  during  the  schocl  year,  the  main 
assessment  sample' was  divided  into  two  randomly  equivalent  subsamples,  one  to 
be  assessed  in  the  winter  and  the  other  to  be  assessed  in  the  spring.  For 
this  purpose,  the  94  PSUs  were  designated  as  winter  PSUs,  spring  PSUs,  or  both 
winter  and  spring  PSUs,  accordi^ng  to  the  scheme  detailed  in  Chapter  3.' 

\^The  bridge  assessments  used  a  subsample  of  the  ^4  PSU.<.  used  for  the  main 
assessment.    The  gi^aSi^-^/age  9  biidge  assessments,  which  were  conducted  in  the 
wf Ater,  used  the  56  PSUs  designated  as  winter  PSUs  in  the  main  assessment;  the 
grade  11/age  17  bridge  assessments,  conducted  in  the. spring,  used  the  56  PSUs 
designated  as  spring  PSUs.     (The  18  largest  PSUs  vere  used  in  both  the  winter 
and  the  spring  assessments.)    The  grade  8/age  13  bridge  assessments,  conducted 
in  the  fall,  used  64  PSUs  selected  from  the  complete  set  of ^4  PSUs  with 
probability  proportional  to  the  measure  of  size  of  the  strata  froip  which  the 
PSUs  were  selected.  .  As  for  the  winter  and  spring  subsajpjiles,  the  18  largest 
certainty  PSUs  wepe  retaineu  in  the  fall  bridge  sample  with  certainty.  Agair, 
Che  scheme  detailed  in  Chapter  3  shows  the  relationship  betveen  these  PSU 
samples,  ^ 


.Stage  2:    Sampling  Schools 


In  the  second  stage*  of  sampling^,  the  public,  private.  Catholic,  Bureau 
of  Indian  Affairs,  and  Department  of  Defense  schools  within  each  of  the  94 
PSOs  were  listed^  a^'-ording  to  the  .tliree  grade/ages.    An  independent  sample  of 
schools  was  selected  separately , for  each  of  the  grade/ages  so  that  some 
schools  tjere  selected  for  assessment  of  two  grade/ages,  and  a  few  were 
selected  for  all  three.    Schools  within  each  PSU  were  selected  (without 
replacement)"  with  probabilities  proportional  to  assigned  measures  of  size. 
(Details  of  the"  p'robabj^Iitles  us'ed 'for  school  selection  appear  in  Chapter  3.) 
Overall*  probfliFil' ties  of  selection  for  high-minority  schools  were  twice  those 
for  other  schools  in  order  to  enlarge  the  sample  of  Black  5.nd  Hispanic 
students,  thereby  ^enhancing  ythe  reliability  of  estimates  for  these  groups. 


The  over-fiil  school  cooperation  rate  exceeded  85  percent  at  each 
•    ,    gradto^'ge.  ^Ti  certain  instances,  refusing  schools  were  replaced  by 
substitutes  accoxtiing  to  the  rules  indicated  in  Chapter  3. 

three  grade/ages,  a  sample  of  schools  was  first  drawn  for  the 
^/      bridge ''assessments  according  the  procedures  detailed  in  Chapter  3.  These 
I  scAoolsT  Vere  then  excltided  from  the  frame  when  the  samples  of  schoc  is  were 

f  drawn  f or*  tb'*   ^ain  assessments.    Appropriate  adjustments  were  made  to  the 

^    sample  weights  for  both  bridge  and  main  s::.mples.     Schools  assigned  main 
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assessment  sessions  were  further  classified  as  belonging  to  the  winter  main 
assessment  or  the  spring  main  assessment. 

Stage  3:    Assigning  Assessment  Sessions  to  Schools 

In  the  third  stage  of  sampling,  assessment  sessions  were  assigned  to  the 
sampled  schools,  as  described  in  section  3.3  of  Chapter  3.    An  assessment 
session  fT>ically  consisted  of  25  to  30  students,  all  of  whcra  could  be 
assessed  following  the  same  procedures.    There  were  two  general  types  of 
sessions  in  the  1988  assessment:  1)  tape  sessions,  where  every  student  was 
administered  the  same  booklet  and  where  audiotape  prompts  paced  the  students 
through  at  least  part  of  the  booklet,  and  2)  print  sessions,  where  a  nuir*_:^r  of 
distinct  booklets  were  administered  and  where  no  audiotape  pacing  was  used. 
(Print  sessions  are  also  called  s^^^ral  sessions,  since  the  assessment  booklets 
were  spiraled  for  administration — see  section  1.4.1.) 

The  assignment  of  sessions  to  schools  was  designed  to  maximize  the 
number  of  session  types  conducted  within  each  PSU,  where  each  session  type 
corresponded  to  a  separate  sample  of  the  population  of  students.     In  most 
sample  schools,  four  types  of  sessions  were  conducted  although  schools  with 
fewer  than  20  eligibles  were  asked  to  conduct  only  a  single  session. 

.age  4:    Sampling  Students 

In  the  fourth  stage  of  sampling,  a  consolidated  list  was  prepared  for 
each  school  of  all  grade- eligible  and  age-eligible  students  for  the  age  class 
for  which  the  school  was  selected.    To  provide  the  target  sample  size,  a 
systematic  selection  of  eligible  students  was  made  from  this  list,  if 
necessary.     In  small  and  medium-s  zed  schools  all  eligible  students  were  in 
the  sample.    For  bridge  (or  trend)  sample  schools  assigned  to  more  than  a 
single  session  type,  students  were  assigned  by  Westat  district  supervisors  to 
print  or  paced- tape  sessions  using  specified  procedures.    A  student  was  not 
assigned  to  more  than  one  session.     Students  assigned  to  paced-tape  sessions 
who  were  not  age-eligible  were  dropped  from  the  assessment. 

Stage  4a:    Excluded  Students 

Some  students  selected  for  the  sample  were  deemed  unassessable  by  school 
authorities  because  they  had  limited  English  language  proficiency,  were  judged 
as  being  mildly  mentally  retarded  (educable) ,  or  were  functionally  disabled. 
For  each  of  these  students,  school  staff  completed  an  excluded  student 
questionnaire,  listing  the  reason  for  exclusion  and  providing  some  background 
information . 
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Stage  4b:    Sampling  Teachers 

The  reading  teachers  of  fourth-grade  students  sa.flpled  for  the  main 
assessments  of  reading  and  the  writing  teachers  of  eighth-grade  students 
sampled  for  the  main  assessments  of  writing  were  identified  and  asked  to 
complete  a  questionnaire  about  the  reading  or  writing  capabilities  of  each 
selected  student  and  about  the  kinds  of  instruction  received. 


Stage  4c:    The  School  and  Principal  Questicnnaires 

A  school  characteristics  and  policies  questionnaire  was  mailed  to  every 
sampled  school  by  Westat  before  the  assessment.    T^.e  Westat  supervisor  then 
collected  the  questionnaires  and  returned  them  to  ETS.    The  schoo"*. 
characteristics  and  policies  questionnaire  is  described  in  Chapter  4. 
The  principal  questionnaire,  distributed  to  the  principal  of  each  sampled 
school  by  Westat  before  the  assessment,  was  used  to  estimate  the  number  of 
grade/ap*- exigible  students  and  to  determine  the  size  and  type  of  community 
used  in  assigning  the  STOC  codes. 


1.3    DEVELOPMENT  OF  ASSESSMENT  OBJECTIVES,  ITEMS,  AND  BACKGROUND  QUESTIONS 

In  1988,  NAEP  conducted  main  assessments  of  students  at  all  three  ages 
in  the  subject  areas  of  reading,  writing,  civics  and  U.S.  history.    Also  part 
of  the  main  assessment  was  a  separately  funded  study  of  geography  achievement 
at  grade  12/age  17  and  a  special  NAEP  study  of  document  literacy  at  grade 
l5/age  13  and  grade  12/age  17.     These  assessments  entailed  the  generation  of  a 
large  number  of  cognitive  items.  In  addition,  a  large  number  of  background  and 
attitude  questions  were  asked  of  students  and  information  was  collected  from 
principals  and  teachers.     Details  of  the  item  development  procedures  followed 
for  the  1988  main  assessment  are  given  in  Chapter  2;  this  section  provides  an 
overview.     (In  addition  to  the  main  assessment,  bridge  studies  were  performed 
in  reading,  writing,  civics,  U.S.  history,  mathematics,  and  science.  Since 
the  instruments  used  for  these  studies  consisted  enti*  ly  of  items  used  in 
previous  assessments,  no  developmental  tasks  were  required  for  their  use  in 
the  1988  assessment.)  ^ 

The  development  of  items  for  each  subject  area  was  supervise^"  by  a 
Learning  Area  Committee  consisting  of  educators,  scholars,  and  citizens 
representing  many  diverse  constituencies  and  points  of  view.     Each  Learning 
Area  Committee  developed  a  set  of  objectives  for  its  subject  area,  proposing 
goals  that  studetits  at  each  grade/age  level  should  achieve.    After  careful  and 
extensive  review,  the  objectives  were  given  to  item  writers  to  develop 
assessxnent  items  to  fit  che  objectives.    Besides  specifying  the  types  of 
cognitive  items  to  be  used  to  measure  academic  achievement,  the  Learning  Area 
Committees  were  also  responsible  for  the  development  of  items  to  measure 
student  backgrounds,  attitudes,  experiences,  and  interests  as  they  relate  to 
the  subject  area. 
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Four  additional  types  of  instruments  were  developed  for  the  1988 
assessment:    a  common  student  background  questionnaire  to  be  given  to  all 
assessed  students  of  a  gi' en  grade/age,  a  school  characteristics  and  policies 
questionnaire,  teacher  questionnaires  for  teachers  of  fourth-  and  eighth  grade 
students,  and  an  excluded  student  questionnaire.     Each  of  these  questionnaires 
was  developed  through  a  broad-basea  consensus  process. 

All  items  in  the  assessment  underwent  extensive  reviews  by  subject  area 
and  measurement  specialists,  as  well  as  careful  scrutiny  to  eliminate  any 
potential  bias  or  lack  of  sensitivity  to  any  group.     Further,  the  items  were 
field  tested  on  a  representative  group  of  students.     Based  on  the  results  of 
the  field  test,  items  were  revised  or  modified  as  necessary  and  then  again 
reviewed  for  lack  of  sensitivity  to  particular  groups.    With  the  help  of  staff 
and  outside  reviewers,  the  Learning  Area  Committee  selected  the  items  to 
include  in  the  assessment. 

Nearly  every  subject  area  included  both  multiple -choice  and  open- traded 
items.    The  exceptions  were  writing,  which  <"onsisted  entirely  of  open-ended 
items,  and  geography  and  science,  %^hich  consisted  entirely  of  multiple -choice 
items.    The  open-ended  items  were  professionally  scored;  the  details  of  the 
professional  scoring  process  are  given  in  Chapter  6.2. 

1.4    ASSESSMENT  INSTRUMENTATION 

Four  types  of  instruments  were  used  in  the  1988  assessment:  student 
assessment  booklets,  excluded  students  questionnaires,  teacher  questionnaires, 
and  a  school  characteristics  and  policies  questionnaire.     This  section 
provides  an  overview  of  these  instruments;  more  information  about  the 
instruments  can  be  found  in  Chapter  4. 

1.4.1    Student  Assessment  Booklets — ^Main  Assessment 

The  student  assessment  booklets  for  the  main  assessment  contained  both 
cognitive  and  Moncognitive  items.  The  total  testing  time  was  approximately  45 
minutes  for  grade  4/age  9  students  and  56  minutes  for  the  older  ages.  A  block 
of  common  bac*cground  que^ Lions  appeared  first  in  eivery  booklet  and  required  10 
minutes  for  completion  for  grade  4/age  9  and  6  minutes  for  the  older  students. 
This  was  followed  by  a  5-minute  block  of  subject-specific  background  questions 
and  (typically)  three  15-minute  blocks  of  cognitive  items  (10-minute  blocks 
for  grade  4/age  9). 

The  assembly  of  cognitive  items  into  booklets  and  their  subsequent 
assignment  to  assessed  students  was  determiiicd  by  a  balanced  i'^complete  block 
(BIB)  design  with  spiraled  administration.    The  first  step  in  implementing  BIB 
spiraling  is  to  divide  the  items  within  a  subject  area  into  units  called 
blocks,  where  each  block  was  designed  to  take  15  minutes  for  the  older 
students  to  complete.    For  the  grade  4/age  9  students,  blocks  requiring  10 
minutes  for  completion  we*e  created.     Some  blocks  were  administered  at  more 
than  one  grade/age;  add? cional  items  were  adued  to  the  end  of  grade  4/age  9 
blocks  which  were  also  administered  to  older  students. 
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These  blocks  wei.e  chen  assembled  into  booklets  containing  the  background 
questions  and  three  blocks  of  subject  area  items  according  to  a  partially 
balanced  incomplete  block  desigii.     (In  a  completely  balanced  incomplete  block 
design,  the  subject  area  blocks  would  be  assigned  to  booklets  in  such  a  way 
that  each  block  appears  in    .\e  sa  .a  number  of  booklets  and  every  pair  of 
blocks  appears  together  in  exactly  one  booklet.    This  is  the  balanced  part  of 
the  method;  the  incomplete  part  refers  to  the  fact  that  no  booklet  contains 
all  items  and  hence  incomplete  data  is  yielded  for  each  assessed  student. 
Such  a  design  allows  the  computation  of  the  correlation  between  each  pair  of 
items  but  generates  a  vast  number  of  differ'*nt  booklets,  particularly  if 
blocks  from  different  subject  areas  are  to  be  paired.) 

In  1988,  the  BIB  design  was  focused — that  is,  each  block  ot  items  within 
each  of  the  subject  areas  was  paired  with  every  other  block  within  that 
subject  area  but  generally,  not  with  blocks  of  items  from  other  subject  areas 
(special  booklets  were  constructed  to  measure  relationships  between  subjects). 
The  focused-BIB  design  used  in  1988  called  for  seven  blocks  of  cognitive  items 
at  a  given  grade/age  to  be  assembled  into  seven  booklets,  where  each  pair  of 
the  seven  blocks  appears  in  exactly  one  booklet  and  where  each  block  appears 
in  three  booklets.     The  focused-BIB  design  also  balances  the  order  of 
presentation  of  the  blocks  of  items  in  the  sense  that  every  block  of  items 
appears  ac  the  first  cognitive  block  in  one  booklet,  as  the  second  block  in 
another  booklet,  and  as  the  third  and  last  block  in  a  third  booklet. 

An  example  of  the  focused-BIB  design  with  seven  blocks  (labeled  A,  B,  C, 
D,  E,  F,  C)  and  seven  booklets  (labeled  Bl  through  B7)  is  as  follows: 


Booklet 

Blocks 

Bl 

A 

B 

D 

B2 

B 

C 

E 

B3 

C 

D 

F 

B4 

D 

E 

G 

B5 

E 

F 

A 

B6 

F 

G 

B 

B7 

G 

A 

C 

In  addition  to  the  focused-B'*B  booklets,  three  tjrpes  of  special  booklets 
were  created  for  the  main  samples.    To  permit  the  calculation  of  correlations 
an  ^ng  items  between  subject  areas,  three  special  inter cor relation  booklets 
containing  one  block  each  of  reading,  U.S.  history,  and  civics  items  were 
created  at  each  grade/age.    Thrse  special  writing  booklets  were  also  created 
at  each  grade/age.     Each  of  these  booklets  consisted  of  a  standard- length 
writing  block  and  a  writing  block  designed  to  take  twice  the  regular  block 
time.     The  purpose  of  the.^e  booklets  was  to  allow  an  evaluation  of  the  effect 
of  the  amount  allocated  time  on  writing  performance.     Finally,  two  dc -ument 
literacy  booklets  were  created  at  grade  8/age  13  ana  grade  12/age  17. 

A  total  of  22  different  booklets  were  assembled  for  grade  ^/age  9,  36 
different  booklets  for  grade  8/age  13,  and  37  different  booklets  fcr  grade 
12/age  17.     These  booklets  were  -hen  spiraled  and  placed  into  bundles. 
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Spiraling  involves  interleaving  the  booklets  in  regular  (systematic)  sequence 
so  that  each  booklet  appears  an  appropriate  number  of  times  in  the  sample. 
Booklets  were  packaged  together  in  bundles  of  25  to  27  booklets,  which  was 
large  enough  to  accommodate  a  typicil  assessment  session.     The  bundles  were 
designed  so  that  each  booklet  woul*'  appear  equally  often  ii  each  position  in  a 
bundle. 

The  final  step  in  the  BIB-spiraling  procedure  is  the  a3signing  of  the 
booklets  to  the  assessed  students.    The  students  within  an  assessment  session 
were  assigned  booklets  in  the  order  in  which  the  booklets  v^ere  bundled.  The 
result  was  that,  typically,  each  student  in  an  assessment  session  received  a 
different  booklet  and,  even  in  schools  with  multiple  sessions,  only  a  few 
2;tudents  received  the  same  booklet  or  block  of  items.     In  the  1988  BIB-spiral 
design,  representative  and  randomly  equivalent  samples  of  about  2,600  grade - 
or  age-eligible  students  responded  to  each  item  (resulting  in  samples  of  about 
2,000  students  eligible  by  age  and  2,000  eligible  by  grade). 

BIB  spiraling  differs  from  the  simpler  matrix  sampling  scheme  used  by 
NAEP  prior  to  198A  to  assign  items  to  students.     In  the  earlier  scheme,  the 
pool  of  items  was  divided  into  distinct  booklets  re';oiring  about       minutes  to 
administer,  and  all  students  withi..  '^n  assessment  session  were  given  the  same 
booklet.     Because  all  students  in  a  session  received  the  same  booklet,  it  was 
possible  to  a,ccompany  the  administration  with  a  paced  audiotape  of  the 
exercise  stimuli,  with  the  aim  of  luinimizing  the  effect  of  a  student's  reading 
ability  on  performance  in  other  subject  areas.    However,  since  each  item 
appeared  in  a  single  booklet,  it  was  impossible  to  estimate  correlations 
between  items  appearing  in  different  booklets.     Furthermore,  the 
administration  of  the  same  items  to  clusters  of  students  within  schools 
results  in  an  increase  in  sampling  variability  over  an  unclustered  sample  of 
the  same  size  because  of  intracluster  correlation. 

The  BIB- spiral  design  permits  the  estimation  of  correlations  b^t/zeen  all 
Items  within  a  subject  area.     Furthermore,  since  the  spiral  design  presents 
each  block  of  items  to  fewer  persons  in  any  school,  but  to  more  schools,  than 
the  simple  matrix  sampling  design,  the  cluster  effect  is  markedly  reduced, 
leading  to  a  sample  with  high  strtistical  efficiency.    The  spiral  design  does 
preclude  the  use  of  audiotape  pacing.     Since  each  student  within  a  session 
responds  to  a  different  set  of  items,  the  instructions  and  the  items 
themselves  must  be  read  by  the  studtnt  as  audiotape  administration  would  be 
unmanageable. 

1.4.2    Student  Assessment  Booklets — Bridge  Samples 

There  were  four  distinct  bridge  sampi.es  in  the  1988  assessment,  each  of 
which  required  the  creation  of  special  booklets' 

Civics  Bridge.    One  booklet  was  created  for  each  of  the  ages  13  and  1 
At  each  age,  the  booklet  consisted  of  a  common  background  block,  a  civics 
background  and  attitude  block  and  three  blocks  of  cognitive  items.  The 
background  blocks  were  from  the  main  assessment;  the  cognitive  blocks 
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contained  items  used  in  previous  assessments  as  well  as  items  used  in  the  1988 
assessment.    The  booklets  were  administered  to  all  students  within  a  session 
using  audiotape  pacing. 

Bridge  to  1984.     Six  booklets  were  used  at  each  of  the  three  grade/age 
classes.    These  booklets  were  identical  to  booklets  used  in  the  1984 
assessments  of  reading  and  writing.    Each  .  )oklet  consisted  of  a  common 
background  block  and  three  cognitive  blocks,  either  two  reading  and  one 
writing  or  one  reading  and  two  writing.    All  cognitive  blocks  also  contained 
subject-related  background  questions.    The  booklets  were  administered  without 
audiotape  and  were  spiraled  through  the  assessment  session. 

Bridge  to  1986  for  Ages  9  and  13.    Three  booklets  were  used  at  each  of 
the  ages  9  and  13  and  were  identical  to  booklets  administered  in  1986.  Each 
booklet  contained  one  block  each  of  reading,  mathematics,  and  science  icems  in 
addition  to  a  common  background  block.    All  cognitive  blocks  also  crntaiued 
subject-related  background  questions.    In  each  session,  all  studeruS  were 
administered  tha  same  one  of  the  three  booklets.    The  mathematics  and  science 
blocks  were  paced  with  an  audiotape;  the  reading  blocks  were  admjLni*:uered 
without  an  audiotape. 

Bridges  to  1986  for  Grade  11/Age  17.     Seven  assessment  booklets  were 
created  for  administration  to  grade  11/age  17  students.     One  booklet  consisted 
entirely  of  blocks  of  U.S.  history  items  from  the  1986  assessment  and  was 
administered  to  the  17[Br86-His]  sample.    The  remaining  six  booklets  consisted 
of  blocks  of  reading,  mathematics,  and  science  items,  were  identical  to 
booklets  administered  in  1986,  and  were  administered  to  the  17[Br86-IUlS] 
sample.    The  booklets  in  both  bridges  were  administered  without  audiotape 
pacing.    Ail  seven  booklets  from  both  bridges  were  administered  to  students  in 
the  same  assessment  session  by  spiraling  through  the  session. 

1.4.3    Other  Instrments 

Besides  the  student  assessment  booklets,  three  other  instruments 
provided  data  relating  to  the  assessment: 

Teacher  questionnaires  were  administered  to  the  reading  teachers  of 
fourth -grade  students  assessed  for  reading  and  to  the  writing  teachers  of 
eighth-grade  students  assessed  for  writing  in  the  main  assessment.  These 
questionnaires  were  designed  to  gather  information  about  the  characteristics 
of  the  teachers  of  the  assessed  students  and  about  the  curricula  and  teaching 
methods  in  the  classroom. 


School  characteristics  and  policies  questionnaires  were  completed  by 
school  principals  or  a  representative  and  provided  information  about  school 
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administration,  staffing  patterns,  special  programs,  subject  requirements,  and 
school  resources. 

Excluded  student  quest iorj:^.elres  were  completed  by  school  personnel  for 
each  sampled  student  excluded  from  the  assessment  and  provided  information 
about  the  reasons  for  exclusion  as  well  as  basic  demographic  characteristics 
of  the  student. 


1.5    FIELD  ADMINISTRATION 

The  field  administration  of  the  1988  assessment  was  the  responsibility 
of  Westat  and  is  documented  in  Chapter  5.    The  field  operation  was  organized 
around  a  staff  at  Westat 's  home  office  and  a  larger  staff  in  the  field.  The 
home  office  staff  consisted  of  a  field  director  and  a  number  of  assistant 
field  directors  who  were  responsible  for  coordinating  all  activities  of  the 
Westat  home  office  staff  related  to  field  operations  and  coordinating 
materials  distribution  and  home-office  receipt  of  assessment  reporting  forms. 
The  field  staff  consisted  of  district  supervisors  and  exercise  administrators. 
The  district  supervisors,  who  were  trained  by  Wes  at,  were  each  responsible 
for  the  assessment  activities  in  one  or  more  PSUs.     Each  district  supervisor 
was  primarily  responsible  for  follow-up  contacts  to  the  school  districts 
containing  schools  selected  for  assessment  (as  explained  below,  ETS  made 
initial  contact),  recruiting  and  training  exercise  administrators  to  work  with 
them  in  administering  the  assessment  sessions,  making  arrangements  for  the 
a.ssessments,  and  selecting  the  sample  of  students  to  be  assessec'  within  each 
school.     The  district  supervisors  and  the  exercise  administrators  administered 
the  assessments,  filled  out  the  necessary  forms,  performed  process  control, 
and  shipped  the  assessment  booklets  and  forms  to  ETS. 

Gaining  school  cooperation  was  the  responsibilit)  of  both  Westat  and  ETS 
staff.     ETS  made  the  preliminary  contacts  preparatory  to  obtaining  school 
cooperation  by  first  contactJ.ng  the  Chief  State  School  Officers,  informing 
them  that  schools  within  the-.r  states  had  been  selected  for  the  assessment 
and,  in  a  later  letter,  listing  the  selected  schools  and  districts.  Later 
mailings  were  sent  to  superintendents  of  public  schools  and  parochial  schools 
and  principals  of  private  schools  for  all  schools  selected  in  the  assessment. 
These  materials  provided  an  explanation  of  NAEP,  a  list  of  the  selected 
schools  in  the  official's  jurisdiction,  and  a  cover  letter  explaining  that  a 
Westat  district  supervisor  would  contact  them  to  set  up  an  introductory 
meeting.    Westat  district  supervisors  then  scheduled  and  conducted 
introductory  meetings,  worked  with  the  schools  to  schedule  the  assessments, 
and,  with  the  exercise  administrators,  conducted  the  assessments.    The  overall 
cooperation  rate  of  schools  originally  selected  for  all  phases  of  the  1988 
assessment  was  86.7  percent.    Farther  detail  on  school  participation  rates 
before  and  after  substitution  is  given  in  sections  3.2  and  3.3  of  Chapter  3. 

The  main  assessment  samples  were  assessed  betwe.-^n  January  4  and 
May  18,  1988  at  all  grade/age  levels.    The  winter  portion  of  the  main 
assessment  and  the  grade  4/age  9  bridge  samples  were  assessed  between  January 
4  and  March  11;  the  spring  portion  of  the  main  assessment  and  the  grade  ll/age 
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17  bridge  samples  were  assessed  between  March  14  and  May  18,  1988.  The  grade 
8/age  13  bridge  sample  was  assessed  between  C^.tober  12  and  December  16,  1987. 

Both  Westat  and  ETS  participated  in  tue  quality  control  of  the  fit  ^d 
administration.    The  quality  control  involved  on-site  visits  by  Westat  and  ETS 
staff  to  verify  the  sampling  of  the  students  and  to  observe  the  conduct  of  the 
assessment  by  the  field  supervisors  and  the  exercise  administrators.    At  the 
end  of  the  assessment,  a  telephone  survey  of  a  10  percent  sample  of  the 
assessed  schools  was  conducted  to  evaluate  the  field  procedures. 

1.6    MATERIALS  PROCESSING  AND  DATABASE  CREATION 

Upon  completion  of  an  as^^ssment  session,  Westat  field  supervisors  and 
exercise  administrators  shipped  the  ..,sessment  booklets  and  forms  from  the 
field  to  ETS  for  entry  into  computer  files,  checking,  and  forming  the 
database.     Careful  checking  assured  that  all  data  from  the  field  were 
received.    More  than  130,000  booklets  or  questionnaires  were  received  and 
processed.     The  extensive  processing  of  these  data  is  detailed  ii,  Chapters  6 
and  6.1. 

Items  requiring  a  written  response  from  the  student  (open-ended  items) 
were  included  in  the  assessment  instruments  for  every  subject  area  except 
geography.     Such  items  had  to  be  professionally  scored;  the  details  of  the 
professional  scoring  appear  in  Chapter  6.2. 

The  transcription  of  the  student  data  into  machine-readable  form  was 
accomplished  by  scanning  the  student  instruments  with  an  optical  scanning 
machine.    An  intelligent  data  entry  system  was  used  for  resolution  of  the 
scanned  data,  entry  of  documents  rejected  by  the  scanning  machine,  and  entry 
of  the  information  on  the  questionnaires.    Additionally,  each  input  datum  was 
checked  to  verify  that  it  was  of  an  acceptable  type,  that  it  was  within  a 
specified  range  or  set  of  ranges  of  values,  and  that  it  was  consistent  with 
other  data  values.    The  entry  and  editing  of  materials  is  discussed  in 
Chapters  6.3  and  6.4. 

Chapter  6.5  discusses  the  extensive  quality  control  checks  undertaken  to 
verify  that  the  information  in  the  database  corresponded  to  the  inform  tion  in 
the  assessment  booklets  and  questionnaires.    A  random  sample  of  each  booklet 
and  questionnaire  was  selected  from  the  database  and  compared  with  the 
original  document.    The  database  was  determined  to  be  quite  free  of  errors 
(with  an  observed  error  rate  of  less  than  .001). 

The  final  step  in  the  preparation  of  the  assessment  data  for  analysis 
was  the  construction  of  the  database  and  the  public -use  data  tapes.    These  are 
described  in  Chapters  6.6  and  6.7. 


Chapter  2 


DEVELOPING  THE  NAEP  OBJECTlViiS,  ITEMS,  AND  BACKGROUND  QUESTIONS 
FOR  THE  1988  ASSESSMENTS  OF  READING.  WRITING,  CIVICS. 
U.S.  HISTORY,  AND  GEOGRAPHY 


Walter  B.  HacDonald,  Ina  V.  S.  Hullis,  Anne  Campbell,  and  Nancy  A.  Mead 

Educational  Testing  Service 


The  subject  areas  constituting  the  1988  assessment  were  specified  by  a 
combination  of  the  legislation  in  place  when  development  work  began  and  the 
advice  of  NAEP's  governing  board  (at  that  time  the  Assessment  Policy 
Committee).^    A  brief  rationale  for  the  inclusion  of  each  subject  area 
follows . 


Reading.     Over  the  past  few  years,  there  had  been  well-warranted  concern 
that  an  alarming  number  of  American  students  were  underprepared  for  the 
literacy  demands  of  an  increasingly  complex  world.     Concern  had  arisen  not 
only  for  student's  intellectual  and  economic  well-being,  but  also  for  their 
ability  to  participate  fully  in  an  information-based  society.     In  addition, 
because  reading  is  central  to  proficiency  in  other  subject  areas,  levels  of 
reading  proficiency  are  likely  to  serve  as  an  overall  barometer  of  educational 
progress. 

Writing.     Good  writing  is  an  essential  underpinning  in  stuc'3nts' 
abilities  to  express  ideas  clearly.      The  importance  of  assessing  writing  was 
exhibited  in  the  NAEP  legislation  effective  through  1988  that  required  ihat  at 
least  once  every  five  years  NAEP  collect  and  report  data  assessing  the  writing 
performance  of  students  at  various  age  or  grade  levels. 

Civics.    The  civics  assessment  took  place  during  the  bicentennial  of  the 
writing  of  the  U.S.  Constitut!,on.     This  was  a  time  of  high  civic  awareness  and 
responsibility  which  focused  attention  on  students'  needs  to  understand 
democratic  principles  in  order  to  appreciate  and  exercise  their  rights  and  to 
recognize  the  responsibilities  inherent  in  being  a  U.S.  citizen.     Civics  was 
first  measured  in  the  context  of  citizenship  assessments  in  1970,  1976,  and 
1982.    The  Assessment  Policy  Committee  selected  civics  because  of  its 


^Although  reading,  writing,  civics,  and  U.S.  history  were  the  four 
subject  areas  approved  for  the  1988  assessment  by  the  Assessir.ent  Policy 
Committee  at  their  October  18-19,  1985  meeting,  the  committee  also  authorized 
NAEP  to  seek  funds  for  a  geography  assessment.    Thus,  the  fifth  subject, 
geography,  was  added  later  when  funding  was  obtained  from  the  National 
Geographic  Society  to  help  support  an  assessment  of  high -school  seniors, 
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Importance  and  because  of  the  time  elapsed  since  it  had  last  been  assessed. 
The  1988  civics  assessment  represents  current  trends  in  civics  education  and 
reflects  the  civics-related  portions  of  past  citizenship  assessments. 


U.S.  History.    History  plays  a  major  role  in  helping  students  understand 
the  world  in  which  they  live.    The  Assessment  Policy  Committee  included  U.S. 
history  in  the  1988  assessment  to  measure  Air'^rican  students'  familiarity  witi. 
the  basic  timeline  and  significance  of  the  main  issues  of  U.S.  history  and  the 
key  roles  American  men  and  women  played  in  effecting  social,  political,  and 
economic  change. 

Geography.    The  extent  to  which  students  understand  geography  affects 
their  ability  to  comprehend  global  relationships  and  preserve  the  world's 
valuable  resources.    An  understanding  of  geography  is  also  pivotal  to  the 
successful  study  of  relat<?d  disciplines,  including  history  and  science.  To 
measure  and  report  on  the  extent  and  quality  of  students'  understanding  of 
geography,  the  National  Geographic  Society  provided  support  for  the  inclusion 
of  geography  in  the  1988  assessment. 


From  its  inception,  NAEP  has  developed  assessments  through  a  consensus 
process  and  the  1988  assessment  was  no  exception.     Educators,  scholars,  and 
citizens  representative  of  many  diverse  constituencies  and  points  of  view 
designed  objectives  for  each  of  the  five  subject  areas,  proposing  goals  they 
felt  students  should  achieve  in  the  course  of  their  education.     After  careful 
reviews,  the  objectives  were  given  to  item  writers,  who  developed  assessi«ent 
questions  appropriate  to  the  objectives.    Al'     uestions  underwent  xtensive 
reviews  by  subject-matter,  measurement,  and  bias/sensitivity  specialists. 
They  were  assembled  and  prin..^d  into  booklets  suitable  for  matrix  sampling  and 
then  administered  by  a  trained  field  staff  to  a  stratified,  multistage 
probability  sample  of  students. 

The  development  for  the  1988  assessment  included  questionnaires  for 
students,  teachers,  and  school  administrators,  as  well  as  a  substantial  number 
of  cognitive  questions  for  each  of  the  five  subject  areas. 

All  1988  development  efforts  were  governed  by  four  major  considerations. 

1)  As  specified  in  the  legislation,  the  objectives  would  be  developed 
through  a  consensus  process  involving  subject  matter  experts, 
school  administrators,  teachers,  and  parents,  and  the  items  would 
be  carefully  reviewed  for  potential  bias. 

2)  As  outlined  in  the  ETS  proposal  for  the  administration  of  the  NAEP 
grant,  the  development  of  objectives  and  items  for  each  subject 
area  would  be  guided  by  a  1  «:rning  Area  Committee. 
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3)  As  described  in  the  ETS  Standards  for  Quality  and  Fairness  (ETS, 
1987),  all  materials  developed  at  ETS  must  be  in  compliance  with 
specified  procedures. 

4)  All  NAEP  items  must  be  submitted  to  a  coinplex  Office  of  Management 
and  Budget  (0MB)  clearance  process  end  all  publications,  including 
objectives  booklets,  submitted  for  review  by  the  Office  of 
Educational  Research  and  Improvement  (OERI) . 

In  general,  developing  the  objectives  and  items  for  the  1988  assessment 
was  a  two-year  proc^o*,,  beginning  in  October  1985  when  the  governing  board 
detemnined  tus  subject  areas  to  be  assessed  and  ending  in  October  1987  when 
data  collection  began  at  age  13  for  the  first  trend  assessment  materials.  The 
schedule  called  for  selecting  the  Teaming  Area  Committees  in  the  fall  of  1987 
and  beginning  objectives  development  in  January  1986.    Once  the  frameworks  for 
the  objectives  were  set,  item  development  proceeded  in  earnest  from  July 
through  November  1986  when  the  clearance  materials  for  the  field  test  were 
submitted  to  the  U.S.  Department  of  Education.    The  field  testing  was 
conducted  in  February  1987.     Subsequent  to  the  xield  testing,  the  Learning 
Area  Committees  met  to  guide  selection  of  the  materials  for  inclusion  in  the 
1988  assessment.     The  materials  for  the  1988  assessment  were  submitted  for 
clearance  in  May  1987.    The  objectives  booklets  were  prepared  for  publication, 
printed,  and  disseminated  during  1987.    It  should  be  noted  that  the  specially 
funded  geography  assessment  of  high  school  seniors  was  on  an  abbreviattd 
schedule,  beginning  in  January  1987,  field  tested  in  April  1987,  and  catching 
up  to  the  main  portion  of  the  assessment  in  time  to  be  administered  beginning 
in  January  1988  together  with  the  reading,  writing,  civics,  and  U.S.  history 
materials.    The  geography  objectives  booklet  was  printed  in  June  1988. 

The  sections  that  follow  in  this  chapter  include  general  overviews  for 
setting  objectives  and  developing  items  and  specific  details  about  developing 
the  objectives  and  the  assessments.     Included  in  Appendix  A  is  a  list  of  the 
more  than  400  consultants  who  participated  in  the  1988  development  process. 


2.1      GENERAL  OVERVIEW  OF  PROCEDUREJ'  FOR  SETTING  OBJECTIVES 

The  general  procedures  followed  for  determining  the  objectives  to  be 
measured  in  ecch  subj^sct  area  in  1988  v/ere  essentially  those  followed  by  NAEP 
in  previous  assessmenf;s  (see  sections  2.3 — 2.6  for  more  detail). 

1)        The  objectives  used  in  the  previous  assessment  were  mailed  to 

about  25  specialists  for  their  review,  comments,  and  suggestions. 
No  constraints  were  placeJ  on  this  activity  and  v/e  asked  for 
candid,  critical  reactions.    The  individuals  involved  in  this 
process  tended  to  be  educators  and  specialists  in  the  field  and 
were  selected  to  represent  differing  points  of  view,  geographical 
locations,  backgrounds,  and  constituencies.    We  sought  advice  from 
a  wide  range  of  sources  for  recommendations  for  this  activity. 
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Learning  Area  Committees  were  established  to  help  guide  assessment 
development  procedures  within  subject  areas.     The  members  of  each 
committee  were  selected  on  the  basis  of  recommendations  from 
professional  organizations,  including  those  related  to  the 
specific  subject  are£,s,  and  with  great  care  to  represent  differing 
perspectives  and  backgrounds.    In  addition  to  gender  and 
race/ethnicity,  care  was  taken  to  have  representation  froji 
classroom  teachers,  district  administrators,  and  state  education 
agencies  as  well  as  researchers  and  university  professors.  As 
much  as  possible,  attention  was  also  given  to  selecting  members 
familiar  with  the  concerns  of  the  professional  organizations 
appropriate  for  each  of  the  five  subject  areas.     These  committee 
members  worked  closely  with  NAEP  staff  in  developing  the  1988 
assessment. 

Comments  from  the  initial  objectives  review  were  synthesized  and 
used  as  input  for  the  first  Learning  Area  Committee  meetings. 

The  first  assignment  of  the  respective  Learning  Area  Committees 
was  to  review  and  revise  the  objectives  booklets  for  each  subject 
area  based  on  their  professional  expertise  and  the  comments  of  the 
previous  reviewers. 

The  new  edition  of  the  objectives  was,  in  turn,  mailed  to 
teachers,  curriculum  specialists  (including  the  state  curriculum 
supervisor  f'     ^ach  subject  area  in  each  of  the  50  y.tates),  and 
school  admin,   trators  practitioners  from  around  the  country. 
These  individuals  are  school  administrators  and  teachers,  as  well 
as  teacher  trainers  who  live  and  work  in  the  practical  educational 
environments.    Their  task  was  to  review  these  objectives  from  the 
point  of  view  of  what  seems  reasonable  and  practical.    As  with 
earlier  steps  in  the  objectives  development  process,  care  was 
taken  to  be  certain  that  appropriate  minority  s^-oup 
representatives  were  included  to  assure  prop€;r  attention  to  these 
sensitivities.    Depending  upon  the  results  of  that  review,  the 
objectives  were  redrafted  with  the  participation  of  the  Learning 
Area  Committee  members  and  others,  as  necessary. 

The  revised  objectives  were  mailed  to  the  Department  of  Education 
for  comment  and  review. 

Further  modifications  of  the  objectives  were  made  as  necessary. 

The  Learning  Area  Committees  completed  the  final  review  of  the 
product. 

The  objectives  were  published,  printed,  and  made  avr^^lable  for 
national  distrioution. 
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2.2      GENERAL  OVERVIEW  OF  PROCEDURES  FOR  DEVELOPING  THE  ITEMS 


A  carefully  developed  and  tested  series  of  steps,  essentially  those 
followed  in  the  past  by  NAEP,  were  used  to  create  test  items  that  reflected 
the  objectives  and  that  measured  achievements  related  to  them  (see  also 
sections  2.3 — 2.6) . 


1)  Each  Learning  Area  Committee  and  the  staff  of  NAEP  determined  what 
spec' Zic  aspects  of  the  objectives  could  be  measured  given  the 
realistic  constraints  of  resources  and  the  feasibility  of 
Implementing  the  measurement  technology.     For  example,  the  grant 
from  the  U.S.  Department  of  Education  set  a  limit  on  the  financial 
resources  available  as  well  as  delineated  the  general  design  of 
the  assessment  as  outlined  in  Chapter  1.    Thus,  the  measures  were 
constrained  to  those  that  could  be  administered  via  paper  and 
pencil  technology  to  groups  of  students.     Each  respective 
committee  made  recommendations  about  priorities  for  the  assessment 
and  types  of  items  to  be  developed. 

2)  The  existing  pool  of  items  to  be  used  to  measure  change  from 
previous  assessments  (trend  items)  was  reviewed  in  detail. 

3)  Item  specifications  were  then  developed  and  prototype  items  were 
created  to  reflect  the  type  of  questions  that  had  been  suggested. 
Trend  items  were  selected. 

4)  Item  writers  with  skills  and  experience  in  creating  items 
according  to  specifications  were  identified  from  both  inside  and 
beyond  ETS  and  scheduled  for  item  development  tasks. 

5)  Newly  created  items  were  reviewed  and  revised  by  staff  and 
external  reviewers. 

6)  Further  language  editing  and  sensitivity  reviews  were  cv  iducted 
according  to  ETS  quality  control  procedures. 

7)  Field  test  materials  were  prepared,  including  the  materials 
necessary  to  secure  0MB  clearance. 

8)  The  field  test  was  conducted  with  a  representative  group  of 
students  from  across  the  country. 

9)  Field  test  booklets  were  scored  and  the  results  analyzed. 

10)  Based  on  these  analyses  and  the  results  of  the  pilot  testing, 
items  were  revised  or  modified  and  re-edited.    They  once  again 
went  through  an  ETS  sensitivity  review. 

11)  With  the  help  of  staff  and  outside  reviewers,  the  Learning  Area 
Committee  selected  the  items  to  include  in  the  assessment. 
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12)      l!:ems  were  assembled  into  ''blocks"  (15-minute  mini- tests)  with 
attention  given  to  balancing  content  coverage  and  difficulty 
levels . 


13)      After  a  final  review  and  check  to  assure  that  each  assessment 

booklet  and  each  block  therein  met  the  overall  guidelines  for  the 
asses IK ""^t,  the  booklets  were  typeset  and  printed. 

2.3      DEVELOPING  THE  READING  ASSESSMENT 

2.3.1  Reading  Objectives 

The  objectives  for  the  1S88  reading  assessment^  carried  forwarc.  the  v^iew 
expressed  in  the  1984  reading  objectives  that  the  processes  of  comprehension 
and  the  extension  of  comprehension  through  interpr  tation  and  analysis  have 
place  in  reading  of  all  kinds  of  texts.     The  1988  objectives  were  based  on  an 
interactive  view  of  reading.     In  this  view,  reading  is  a  dynamic  process  in 
which  a  number  of  elements  interact,  including  the  reader,  the  material  being 
read,  the  purposes  of  'Jr.e  reading,  the  re^-der's  previous  experiences,  and  tlie 
context  for  reading. 

The  first  objective,  "Comprehends  What  Is  Read,"  is  central  to  the 
reading  process.    This  obje-ztive  encompar.ses  the  comprehension  of  var'.ous 
types  of  written  materials  us  well  a.s  the  comprehension  of  materials  read  for 
a  particular  purpose.    Objective  two,  "Extends  Comprehension,"  involves 
deliberate  kinds  of  analysis,  interpretation,  and  evaluation.  Objective 
three,  "Manages  the  Reading  Experience,"  recognizes  that  good  reade  s  develop 
a  variety  of  strategies  to  help  them  comprehend  what  they  rer-A.  These 
strategies  include  using  the  structure  and  organization  of  the  text;  using 
readers'  aids;  showing  flexibility  in  one's  approach  t<  reading;  and  .<ielecting 
reading  materials  appropriate  to  the  purpose.     The  fourth  objective,  "Values 
Reading,"  discusses  the  kinds  of  appreciation  that  students  can  develop.  The 
values  include  reading  as  a  source  of  enjovioent;  reading  to  expand 
understanding  and  to  fulfill  personal  g«   \s;  reading  as  a  means  of  acquiring 
knowledge  and  lemming  new  skills;  and  recognizing  the  cultural  role  of 
written  language.     In  order  to  help  teachers  using  the  objectives, 
instructional  strategies  for  each  of  the  obj  actives  v/ere  included  in  the 
booklet.     In  addition,  because  the  1984  reading  assessment  was  so  extensive  in 
content  coverage  and  because  the  proficiency  scale      .'eloped  from  it  forms  the 
basis  for  reporting  data  through  the  1980s,  a  briei  -description  of  the  reading 
scale  was  included. 


^Reading  Objectives ,  1986  and  1988  Assessments    (Princeton,  NJ: 
Educational  Testing  Service,  National  Assessment  of  Educational  Progress,  Jui.e 
1987). 
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2.3.2  Reading  Materials 


As  with  previous  NAEP  reading  assessments,  a  variety  of  reading  stimulus 
materials  were  used  that  reflected  the  realities  of  reading  passages,  charts, 
and  instructions  found  in  texts,  newspapers,  and  source  documents.    To  this 
end,  materials  were  drawn  from  many  sources,  including  science  and  social 
studies  textbooks,  peer  writing,  technical  writing,  and  directions. 

2.3.3  Reading  Item  Development 

The  main  goals  of  the  1988  reading  assessment  were  to  provide  a  link  to 
th3  1984  reading  assessment  and  to  broaden  the  scope  of  the  assessment  with 
some  new  items.    As  a  result,  the  items  from  the  1984  assessment  became  the 
foundation  for  the  1988  assessment  and  the  extensive  development  of  new  items 
was  determined  to  be  unnecessary.    New  items  were  developed  under  guidelines 
that  included  such  factors  as  type  of  material  x'narrative,  expository,  or  peer 
writing),  number  of  features  being  matched,  number  of  possible  distractors, 
and  correspondence  between  the  question  and  text.     These  guidelines  were 
helpful  in  assembling  the  items  for  the  field  test,  particularly  in 
determining  an  approximate  difficulty  level  for  the  items.     In  assembling  the 
items  into  blocks  for  the  field  test,  factors  that  were  taken  into 
consideration  included  the  length  of  the  stimulus,  the  type  of  reading 
material,  and  the  estimated  difficulty  of  the  items. 

Once  the  field  test  data  had  been  analyzed,  the  reading  Learning  Area 
Committee  met  to  select  trend  items  from  the  1984  reading  assessment  and  to 
select  new  items  that  //ould  complement  the  pool  of  trend  items.    Trend  items 
were  selected  with  careful  attention  to  their  psychometric  characteristics  as 
evaluated  in  prior  assessments.    The  new  items  were  selected  on  the  basis  of 
their  statistical  item  analyses  from  the  field  test  as  well  as  the  type  of 
stimulus  material.    One  factor  that  had  to  be  taken  into  consideration  in 
compiling  the  items  into  blocks  was  that  at  least  one  trend  and  one  new  block 
at  each  age  level  had  to  overlap  with  the  age  level  above  and/or  below  it. 
There  were  to  be  at  least  three  blocks  of  trend  items  at  each  age  level  as 
well.     In  addition  to  the  criteria  for  overlap  acroi>s  age  levels,  trend  blocks 
were  selected  tc  represent  the  best  content  coverage  across  the  objectives  and 
be  representative  of  the  range  of  performance  in  previous  assessments. 

Once  the  items  were  selected  and  the  blocks  assembled,  they  underwent 
final  review  by  ETS  subject-area  specialists  and  ».est  editors  as  well  as  a 
review  to  detect  the  presence  of  any  bias  according  to  the  ETS  Standards  for 
Quality  and  Fairness  (ETS,  1987).    After  internal  review,  the  OMB  clearance 
package  was  prepared  and  submitted. 

2.3.4  Reading  Background  and  Attitude  Questions 

The  1984  and  1986  assessment  provided  a  wide  range  of  reading  background 
questions  for  consideration  for  the  1988  assessment.    The  Learning  Area 
Committee  reviewed  this  pool  of  items  and  selected  a  number  Tor  reuse.  Of 
continuing  interest  were  items  that  measure  student  behaviors  that  indicate 
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interest  in  reading  (reading  for  fun  in  your  spare  time,  telling  a  friend 
about  a  good  book).    Other  questions  from  prior  years  included  spare  time 
spent  read-ug  fiction  and  nonfiction  and  the  student's  evaluation  of  his  or 
her  own  competence  as  a  reader. 

The  Learning  Area  Committee  was  interested  in  expanding  questions  about 
the  home  reading  environment:    whether  students  subscribe  to  magazines  at  hoTie 
or  have  books  of  their  own,  whether  students  were  read  to  when  they  were 
young,  whether  students  read  aloud  to  someone  at  home.    Also  added  to  the 
assessment  were  questions  about  frequency  of  various  teaching 
practices — introducing  the  main  idea  of  a  story,  pointing  out  new  vocabulary, 
giving  students  a  list  of  questions  to  answer.     Finally,  the  Learning  Area 
Committee  wanted  to  know  how  often  students  used  various  reference  books,  such 
as  dictionaries  and  encycloped^  . 


2.4  DEVELOPING  THE  WRITING  ASSESSMENT 

2.4.1  Writing  Objectives 

'Hie  objectives  for  the  1988  writing  assessment^  were  essentially  a 
revised  update  of  the  objectives  developed  for  the  1984  writing  assessment. 
Because  a  major  development  effort  was  expended  in  the  preparation  of  the  1984 
objectives,  NAEP  anticipated  that  those  objectives  still  reflected  current 
theory  of  writing.    To  make  sure,  however,  that  the  objectives  for  the  1988 
assessment  were  up-to-date,  NAEP  sent  the  1984  objectives  to  teachers  and 
theorists  across  thft  nation  and  asked  them  to  comment  on  and  revise  the 
objectives  as  they  felt  necessary.    Although  many  reviewers  suggested  numerous 
small  revisions,  the  reviews  supported  the  hypothesis  that  extensive  revisions 
were  unnecessary.    The  one  major  concern  was  integrating  the  concept  "learning 
through  writing**  across  all  writing  purposes — informative,  persuasive,  and 
personal/imaginative  narrative — rather  than  segregating  it  as  a  single 
obj  ective. 

NAEP  staff  collated  the  conmients  and  prepared  a  revised  version  of  the 
objectives.     This  revised  version  was  sent  to  members  of  the  Learning  Area 
Committee  prior  to  their  first  meeting.    When  the  committee  met,  they 
discussed  the  reviews  and  the  revised  objectives  and  then  each  member  took  a 
section  and  revised  it  further  to  reflec:  the  discussion.    Their  revisions 
became  the  basis  for  the  final  version  of  the  objectives. 

The  writing  objectives  were  based  on  the  premise  that  individuals  write 
for  a  purpose  to  an  audience.    Reflecting  this  premise,  the  first  objective 
was  that  students  use  writing  to  accomplish  a  variety  of  purposes: 
informative,  persuasive  and,  personal/narrative.    The  second  objective — that 
students  manage  the  writing  process — focused  on  the  importance  of  the  process 
that  leads  to  a  piece  of  writing.    The  third  objective — that  students  control 
the  forms  of  written  language — concerned  itself  with  such  skills  as 


^Writing  Objectives ,  1988  Assessment  (Princeton,  NJ:     Educational  Testing 
Service,  National  Assessment  of  Educational  Progress,  August  1987). 
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organizing,  elaborating,  and  using  appropriately  the  conventions  of  writing 
(usage  and  mechanics).    The  fourth  objective  was  that  students  value  writing 
and  what  has  been  written  and  so  underscored  the  importance  of  understanding 
the  value  of  writing  and  the  roles  written  works  serve  in  our  society.  In 
order  to  help  people  put  the  objectives  into  practice,  ideas  and  suggestions 
for  teaching  to  each  objective  were  presented  in  the  final  section  of  the 
objectives  booklet. 


2.4.2  Writing  Item  Development 

Because  of  the  similarity  between  the  1984  and  1988  objectives  and 
because  a  major  effort  had  also  been  put  forth  to  develop  the  tasks  for  the 
1984  writing  assessment,  NAEP  felt  that  those  items  could  be  the  foundation 
for  the  1988  assessment.     In  addition,  NAEP  had  access  to  the  many  tasks  being 
developed  for  the  California  Assessment  Program  (CAP).     For  these  reasons,  it 
was  possible  to  keep  the  new  item  development  effort  to  a  minimum. 

First,  the  Learning  Area  Committee,  made  up  of  seven  writing 
specialists,  evaluated  and  selected  the  1584  items  by  taking  into 
consideration  such  factors  as  data  from  the  1984  writing  assessment,  the 
purpose  for  writing  addressed  by  the  task,  the  appropriateness  of  the  items 
for  more  than  one  age,  and  the  need  to  use  enough  items  to  provide      bridge  to 
the  1984  assessment.    The  items  that  the  Learning  Area  Committee  selected 
became  the  pool  for  the  writing  trend  bridge  assessment  and  were  the 
foundation  of  the  pool  of  items  for  the  more  comprehensive  1988  main 
assessment.    After  selecting  the  1984  trend  items,  the  committee  then 
considered  what  other  items  were  needed  to  meet  the  specifications  for  the 
1988  main  assessment,  reviewed  the  materials  from  the  CAP,  and  developed  new 
items.     The  developers  of  the  CAP  writing  assessment  program,  which  is  bu-.^t 
on  a  theoretical  base  very  similar  to  that  used  by  NAEP,  had  worked  very  long 
anJ  hard  to  develop  over  80  grade  8  prompts  and  were  pleased  to  share 
materials  with  the  NAEP  committee.    We  are  particularly  grateful  for  access  to 
this  resource  and  indebted  to  CAP.     In  developing  th'  new  prompts  for  the  1988 
assessment,  particular  attention  was  given  to  making  the  writing  tasks 
relevant  to  students  at  the  grade  level  being  assessed.     Consistent  with  the 
growing  emphasis  on  process-writing  instruction,  it  was  also  determined  to 
study  the  effect  on  performance  of  the  length  of  time  provided  to  respond  to 
the  writing  tasks.    A  small  study  on  this  issue  co.-ducted  in  conjunction  wich 
the  South  Carolina  Department  of  Education  had  indicated  Sv-me  improvements  in 
performance.    Therefore,  some  tasks  were  identified  to  be  given       students  in 
two  versions.     In  the  regular  version,  fourth-grade  students  had  10  minutes  to 
respond  and  eighth-  and  twelfth-grade  students  had  15  minutes  to  respond.  In 
addition,  some  of  the  tasks  were  also  given  with  longer  response  times — 20 
minutes  for  fourth  graders  and  30  minutes  for  students  in  the  upper  grades. 
(Because  any  assessment  context  seriously  J^^tricts  provision  for  the  usual 
revision  strategies,  including  peer  review  and  outside  consulting  reqources, 
in  either  situation  the  responses  were  to  be  viewed  as  first  draft  writing  and 
evaluated  accordingly.) 

Once  the  pool  of  prompts  had  been  developed,  thes.   itr.ns  were  reviewed 
by  ETS  subject-area  specialists,  sensitivity  reviewers,  and  test  odirors. 
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Subsequent  to  the  internal  revie.  procedures,  NAEP  staff  submitted  the  items 
for  OERI  and  0MB  clearance  prior  to  field  testing. 

After  the  field  test,  the  Learning  Area  Committee  met  once  again  to 
review?  the  results  and  to  select  the  new  items  that  would  complete  the  item 
pool  for  the  1988  assessment.    Tliose  items  underwent  final  review  by  ETS 
subject-area  specialists  and  test  editors  as  well  as  a  review  to  detect  any 
bias  according  to  the  ETS  Standards  for  Quality  and  Fairness  (£TS,  1987).  The 
final  item  pool,  including  the  trend  items,  was  then  submittea  for  0MB 
clearance  for  the  1988  assessment. 


2.4.3    Writing  Background  and  Attitude  Questions 

For  r  asons  explainer*  in  Chapter  1,  the  overall  NAEP  design  between  1984 
and  1988  chi^iged  from  one  where  student  background  questions  wc*.e  BIB-spiraled 
across  the  assessment  along  with  cognitive  items  to  one  where  all  students 
assessed  in  writing  at  a  given  grade  level  were  given  the  same  five -minute 
writing  background  questionnaire.    Again,  as  the  result  of  the  c  .tensive 
development  effOi.c  for  the  1984  writing  assessment  and  also  as  a  result  of  the 
design  used  in  1984,  that  assessment  had  contained  over  100  background 
questions  and  all  had  been  thoroughly  analyzed  and  reported  in  2'he  Writing 
Report  Card:    Writing  Achievement  in  American  Schools  (Applebee,  Langpr,  & 
Hullis,  1986),     This  pool  provided  an  ample  source  of  questions  fr^  devising  a 
five-minute  background  questionnaire  for  each  of  the  three  grade  levels  (10 
questions  at  grade  4,  24  questions  at  grade  8,  and  35  questions  at  grade  12). 

The  Learning  Area  Committee  reviewed  the  questions  and  the  1984  results 
and  determined  which  areas  to  carry  forward  into  the  1988  assessment.  The 
reduction  in  background  coverage  for  198^  necessitated  difficult  decisions, 
but  agreement  was  reached  about  the  need  to  focus  on  information  about 
students'  use  of  and  instruction  in  the  writing  process  as  well  as  the  amount 
of  writing  actually  done  by  students. 

The  latter  two  areas  of  emphasis  were  particularly  important  in  view  of 
the  teacher  questionnaire  that  was  developed  for  the  writing  teachers  of  the 
eighth- grade  studencs  who  participated  in  the  assessment  (see  section  2.7  for 
further  detail  about  the  teacher  questionnaires) .    While  both  students  and 
teachers  were  asked  some  questions  relevant  to  their  jwn  experiences,  care 
also  was  given  to  obtaining  a  coordinated  set  of  information  from  teachers  and 
students.     In  this  way,  NAEP  was  able  to  obtain  the  perspectives  of  both 
teachers  and  students  about  writing  instruction  and  compare  the  two  (see 
Learning  to  Write  in  Our  Nation's  Schools:    Instruction  and  Achievement  in 
1988  at  Grades  4,  8,  and  12  (Applebee,  Langer,  Mullis,  Jenkins,  &  Foertsch, 
1990). 


2.5    DEVELOPING  THE  CIVICS  AND  U.S.  HISTORY  ASSESSMENT 

Because  state  education  agencies  had  expressed  a  desire  to  have  greater 
input  into  the  NAEP  assessment  development  process  so  that  state  curriculum 
concerns  could  be  adequately  reflected  in  the  discussions  about  the  objectives 
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and  the  content  of  the  assessment,  the  civics  and  U.S.  history  assessments 
were  developed  together  using  a  new  model — one  Learning  Area  Committee  with 
the  additional  support  of  a  State  Advisory  Committee.    The  integrated 
development  of  the  civics  and  U.S.  history  would  also  be  a  very  efficient 
protocol  for  development  by  decreasing  the  redundarcy  of  particular  aspects  of 
these  related  subject  areas.    Th'.s  model  for  development  was  accepted  by  OERl 
and  ratified  by  the  Assessment  Policy  Committee  at  their  May  30-31,  1986 
meeting.     In  November  1985,  ^'AEP  staff  solicited  recoiuTuendations  for  reviewers 
and  Learning  Area  Committee  membership  fr^     the  50  State  Testin^  'rectors. 

To  enhance  state  participation  in         actual  development  of  the  civics 
and  U.S.  histoty  assessment*  a  plan  was  developed  with  the  State  Testing 
Directors  at  the  Large  Scale  Assessment  Conference  held  in  Boulder,  Colorado, 
June  9-12,  1986.     The  plan  established  a  State  Advisory  Committee  to  work  in 
conjunction  with  the  Learning  Area  Committee. 

2.5.1  Civics  and  U.S.  History  Objectives 

The  eight-member  civics  and  U.S.  history  Learning  Area  Committee  first 
met  in  July  1986  to  draft  the  objectives  and  suggest  prototype  items.  Unlike 
reading  and  writing  with  ongoing  development  and  extensive  efforts  in  recent 
assessments,  civics  had  not  been  assessed  since  1976  (as  part  of  citizenship) 
and  although  U.S.  history  had  been  assessed  in  1986,  that  assessment  was  a 
knowledge-based  assessment  at  only  grade  ll/rge  17.    Thus,  more  initial  work 
was  needed  to  update  and  elaborate  on  the  existing  objectives  for  civics  and 
U.S.  history. 

For  civics,  a  three-dime-^ 'ional  matrix  made  up  of  content,  co.itext,  and 
cognition  frame  three  broad  objectives.^    The  content  dimension  included  the 
democratic  principles  and  the  purpose  of  government;  the  political 
institutions;  tie  political  process;  and  rights,  responsibilities,  and  the 
law.    Each  of  tue  areas  were  further  broken  down  into  numerous  assessment 
topics.     The  concext  dimension  included  home,  school,  community,  state, 
nation,  and  world.    These  addressed  the  expectations  that  civics  learning 
begins  in  dildhood,  continues  through  adolescence  and  schooling,  ^ind  matures 
in  adulthood  when  people  are  participating  in  society.    The  cognition 
dimension  included  1)  knows  and  2)  understands  and  applie'-.     These  abilities 
addressed  the  notion^;  that  students  must  first  be  able  to  recognize  factual 
knowledge  and  then  be  able  to  interpret  information  and  be  aware  of  how 
concepts  and  facts  are  interrelated. 

As  with  the  civics  objectives,  the  U.S.  history  objectives^  took  form  as 
a  matrix  framed  by  three  broad  objectives.    The  first,  chronology  of  events, 
persons,  and  documents,  included  the  following  eight  histv'^rcal  periods: 


Civics  Objectives,  1988  Assessment  (Princeton,  N J .     Educational  Testing 
Service,  National  Assessment  of  Educational  Progress,  September  1987). 

^U.S.  History  Objectives,  1988  Assessment  (Princeton,  N J :  Educational 
Testing  Service,  National  Assessment  of  Educational  Progress,  August  1987). 
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1)  Exploration  and  colonization  up  to  1763;  2)  The  revolutionary  era,  the 
Constitution,  and  the  new  Republic,  1763-1815;  3)  Economic  and  social 
developmenc  of  *-he  Antebellum  Republic,  1790-1861;  4)  Crisis  of  the  union: 
origins  of  the  v  ii ,  the  war,  and  reconstruction,  1850-1877;  5)  The  rise  of 
modem  America  and  World  War  I,  1877-1920;  6)  The  United  States,  1920-1941;  7) 
World  War  II  and  the  postwar  era,  1931-1968;  and  8)  Modern  post- industrial 
era:  1968  to  the  present.    Additional  topics  were  specified  under  each  of 
these  chronological  periods.    The  second  objective,  historical  contexts, 
included  the  following  contextual  backgrounds,  political  life;  economic  life, 
cultural,  social,  and  family  life;  and  intellectual  life.    Reasoning  skills 
formed  the  third  dimensi'^n  of  the  framework.     The  first  skill,  reference 
skills  and  knowledge,  addressed  the  need  to  know  historical  aspects  and  the 
need  to  know  how  to  expand  an  existing  knowledge  base.    The  second  skill 
addressed  the  need  for  an  understanding  and  a  comprehension  of  the  association 
of  ideas  and  the  perception  of  relationships. 

In  September  1986,  the  State  Advisory  Committee  was  convened  to  review 
these  draft  objectives  and  to  review  items  submitted  by  states  for  inclusion 
in  the  civics  and  U.S.  history  assessment.    For  both  the  civics  and  U.S. 
hi«:tory  objectives,  the  committee  sugj^ested  minor  improvements  to  revise  or 
rephrase  some  of  the  topic  areas.    Following  the  review  of  the  objectives,  the 
coimnittee  weighted  each  of  the  topic  area  with  consideration  to  their 
relevance  at  a  given  grade/age  level.    The  revised  objectives  were 
subsequently  reviewed  by  the  Learning  Area  Coinraittee  who  were,  by  and  large, 
satisfied  with  the  results. 


2.5.2  Civics  and  U.S.  History  Iten  Development 

Some  of  the  new  items  used  in  the  1988  assessment  were  submitted  by  some 
of  the  states;  others  were  developed  by  either  ETS  staff  or  outside  item 
writers.    To  facilitate  the  writing  of  new  items  that  were  appropriate  to  the 
curriculum,  1*^  social  studies  teachers  were  convened  at  an  item  development 
conference  at  ETS  to  write  and  revise  new  items.     Following  this  conference, 
additional  input  was  obtained  from  another  15  social  studies  teachers  who 
reviewed  items  and  suggested  other  new  items  during  an  item  development 
conference  at  the  Social  Science  Education  Consortium  in  Boulder.    The  item 
development  process  proceeded  through  the  fall  of  1986  in  accordance  with  the 
guidance  provided  by  the  Learning  Area  Committee.     The  new  items  were  reviewed 
by  subject-matter  specialists,  edited,  and  presented  to  the  committee.  The 
condiiittee  met  in  October  1986  to  review,  revise,  and  select  items  for  field 
testing  in  the  spring.    To  ensure  the  appropriateness  of  the  items  selected 
for  stv dents  in  the  fourth  grade,  these  items  were  spnt  to  eight  fourth- grade 
teachers  for  review  in  November  1986.    All  materials  were  submitted  to  OERI  in 
November  1986,  and  revised  and  resubmitted  in  DecembeiT  to  OERI  for  submission 
to  0MB.    After   "ield  testing,  the  Learning  Area  Committee  met  for  a  third 
time,  in  April  i.987,  to  review  the  field  test  results  and  select  the  items  for 
the  1988  assessment  of  civics  and  U.S.  history.    Consistent  with  routine 
procedures,  the  items  selected  for  the  1988  assessment  were  subypitted  for 
internal  ETS  review  and  submitted  for  OERI/OMB  clearance. 
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2.5.3    Civics  and  U.S.  History  Background  and  Attitude  Questions 


The  civics  and  U.S.  history  Learning  Area  Comi..ittee  was  primarily 
concerned  with  gathering  information  about  students'  instructional 
experiences.     At  all  grade/age  levels,  students  were  asked  how  much  they  had 
studied  the  various  topics  covered  in  the  assessment.    At  the  two  higher 
grade/age  levels,  students  were  asked  about  the  amount  of  course  work  they  had 
taken  and  their  teachers'  practices.    High- school  students  also  were  asked 
about  their  homework,  their  grades,  end  their  attitudes  toward  the  subject 
areas. 


2.6      DEVELOPING  THE  GEOGRAPHY  ASSESSMENT 

The  1988  geography  assessment  was  initiated  and  funded  by  the  National 
Geographic  Society  due  to  well-warranted  concerns  about  the  state  of  geography 
learning  in  the  United  States.    The  development  of  this  assessment  did  not 
start  at  the  same  time  as  the  oUier  subjects,    llie  design,  planning,  and 
funding  for  a  geography  assessment  of  grade  12/age  17  students  commenced  man 
raonths  later  in  NAEP's  1988  development  cycle. 

A  five-member  Learning  Area  Committee  was  created  with  individuals 
recommended  by  the  National  Geographic  Society  in  accordance  with  the 
guidelines  for  committee  membership  articulated  in  section  2.1.  This 
committee  eagerly  took  the  charge  to  develop  an  assessment  in  a  new  subject 
area  never  before  assessed  by  NAEP.    The  challenge  of  a  late  start  and  a  brand 
new  subject  area  for  assessment  was  balanced  by  the  modest  scope  of  this 
assessment  relative  to  the  other  subject  areas — one  grade  and  three  blocks  of 
questions  versus  three  grades  with  approximately  seven  blocks  of  questions  per 
grade . 

2.6.1  Geo<^raphy  Objectives 

The  geography  Learning  Area  Committee  first  convened  at  ♦ihe  end  of 
January  1987.    At  this  meeting,  the  objectives  of  the  1988  geography 
assessment^  were  formulated  to  reflect  the  current  trends  in  geography 
education.    During  this  process,  the  Wisconsin  geography  assessment  provided 
valuable  background  information  upon  which  to  build  NAEP's  draft  framework  for 
a  national  assessment  of  geography.    This  framework  was  organized  around  three 
dimensions  of  geography:  geographic  skills  and  tools,  including  the  use  of 
maps,  charts,  and  globes;  geographic  knowledge  and  concepts,  including  the 
understanding  the  area  of  physical  and  cultural  geography;  and  geographic 
inquiry,  including  the  cognitive  application  of  skills,  knowledge,  and 
understanding  to  new  situations.    The  dimension  of  geographic  knowledge  and 
concepts  (content)  was  divided  into  physical  and  cultural  geography.  Physical 
geography  included  physical  locations,  places,  and  regions;  climatology  and 
meteorology;  and  the  evolution  of  land- form  features  of  the  earth's  surface, 


^Geography  ObjecCives ,  1988  Assessment  (Princeton,  N J :  Educational 
Testing  Service,  National  Assessment  of  Educational  Progress,  June  1988). 
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whereas  cultural  geography  included  cultural  locations,  places  and  regions, 
human  ir^pacts  on  the  environment;  influences  of  the  environment  on  human 
activity;  and  spatial  interactions.    All  of  the.  categories  within  physical  and 
cultural  geography  were  broken  down  into  numerous  subcategories  to  help 
characterize  topics  for  assessment 

Following  the  Learning  Area  Committee  meeting,  the  draft  objectives  were 
reviewed  by  external  consultants  representing  various  constituencies  and 
revised  as  necessary.     Because  of  the  hastened  development  time  and  also 
because  of  their  keen  interest,  the  Learning  Area  Committee  remained  closely 
involved  throughout  the  review  and  revision  process. 

2.6.2  Geography  Item  Development 

Following  the  drafting  of  objectives,  item  development  proceedr^d  at  an 
accelerated  pace  in  order  to  bring  the  geogra^^hy  assessment  into  phase  with 
the  assessments  in  reading,  writing,  civics,  and  U.S.  history.    About  IOC 
cognitive  items  were  written  by  ETS  test  development  staff  and  100  items  were 
written  by  members  of  the  Learning  Area  Committee.     These  items  were  reviewed 
by  about  10  reviewers  and  used  to  assemble  six  blocks  of  cognitive  items.  The 
rapid  preparation  did  not  preclude  the  standard  ETS  test  development 
procedures  to  ensure  quality  and  fairness  of  the  geography  item  blocks. 

The  geography  field  test  data  were  gathered  and  analyzed  in  preparation 
for  a  May  1987  meeting  of  the  Learning  Area  Committee.    At  this  meeting,  the 
committee  reviewed  the  items  and  their  statistics  to  select  a  pool  of  items 
that  would  be  appropriate  for  the  assessment  of  the  geography  knowledge  and 
skills  possessed  by  students  at  grade  12/age  17.     Decisions  were  monitored  in 
terms  of  the  coverage  of  both  the  four  content  areas  and  the  importance  of 
assessing  conceptual  understanding  as  well  as  facts.    After  the  meeting,  test 
developers  assembled  one  test  booklet  comprising  one  five* minute  block  of 
general  background  questions,  one  five -minute  block  of  geography  background 
questions,  and  thrae  IS-minute  blocks  of  cognitive  items,  totaling  78  items  in 
all.    These  materials  were  reviewed  thoroughly  by  ETS  specialists  and 
submitted  for  0MB  clearance. 


2.6.3    Geography  Background  and  Attitude  Questions 

As  mentioned  above,  geography  assessment  development  also  included  a 
block  of  background  questions  for  inclusion  into  a  test  booklet  with  the 
cognitive  blocks.     In  developing  materials  for  the  student  questions  specific 
to  geography,  the  Learning  Area  Committee  focused  on  coursework  and  content 
coverage.    Students  were  asked  what  geography  courses  thiy  had  taken  du*:ing 
high  school  and  how  much  they  studied  the  various  topics  covered  by  the 
objectives  and  assessment  items. 
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2./      DEVELOPING  THE  COMMON  CORE  STUDENT  QUESTIONS  AND  QUESTIONNAIRES 


Additional  instruments  were  developed  for  the  1988  assessment:  a  common 
core  of  student  background  questions  containing  21  questions  at  ^ra     4,  21 
questions  at  grade  8,  and  33  questions  at  grade  12;  teacher  questic  naires 
containing  56  questions  at  grade      and  67  questions  at  grade  8;  school 
characteristics  anu  policies  questionnaires  containing  91  questions  at  grade 
4,  94  questions  at  grade  8,  and  107  questions  at  grade  12;  and  an  excluded 
student  questionnaire  containing  28  questions. 

The  student,  teacher,  and  school  instruments  were  designed  tc  collect 
information  about  home,  classroom,  and  school  factors  related  to  four  policy 
issues  that  had  also  been  the  focus  of  the  1986  assessment:  instructional 
practice,  students  at  risk,  teacher  workforce,  and  effective  schools. 

The  development  of  policy  issues  and  items  was  an  iterative  process  that 
involved  staff  work,  field  testing,  periodic  review  by  an  external  advisory 
group,  and  review  by  the  Assessment  Policy  Committee.    At  an  initial  meeting, 
a  group  of  external  consultants  drafted  a  set  of  policy  issues  and  made 
recommendations  regarding  the  design  of  the  assessment.    They  were 
particularly  interested  in  capitalizing  on  the  unique  properties  of  NAEP  and 
not  duplicating  the  purposes  underlying  other  surveys  (e.g..  The  National 
Survey  of  Public  and  Priva,:e  School  Teachers  and  Administrators  and  The 
National  Education  Longitudinal  Study).     They  recommended  a  more  focused  study 
of  the  relationship  between  student  achievement  and  instru<^tional  practices. 
This  recommendation  provided  the  impetus  for  a  major  redesign  of  the  teacher 
questionnaire  (see  below).     Items  were  developed  to  assess  the  policy  issues 
and  field  tested  with  students,  teachers,  and  principals.    The  policy  issues, 
items,  and  field  tert  results  were  reviewed  by  the  group  of  external 
consultants  who  id'  ntif ied  specific  items  to  be  included  in  the  final 
questionnaires.    The  field  test  results  and  the  recommendations  of  the 
consultants  were  also  reviewed  by  the  Assessment  Policy  Committee.     The  ittms 
were  then  assembled  into  questionnaires  and  submitted  to  internal  ETS 
procedures  to  ensure  fairness  and  quality.    The  background  questior*nair<.b  were 
submitted  for  0MB  clearance  together  with  the  cognitive  items. 

Every  student  booklet  began  with  a  common  core  of  background  questions. 
In  many  cases  the  questions  used  in  1988  were  taken  from  prior  assessments. 
Although  many  of  the  questions  were  common  to  the  three  grade/age  levels 
as    ssed,  some  were  specifically  targeted  to  elementary  or  high-school 
students.    At  grade  4/age  9,  the  background  questions  were  read  aloud  to  the 
students  and  took  approximately  10  minutes  to  complete.    At  the  other  two 
grade/age  levels,  only  the  first  race/ethnicity  question  was  read  aloud, 
students  read  and  answered  the  remaining  questions  on  theit  own  during  a  five 
minute  time  period.    The  questions  asked  about  demographics,  home  environment, 
and  instruction  experiences. 

The  teacher  questionnaire  was  administered  to  samples  of  grade  4  and 
grade  8  teachers.    All  grade  4  students  assessed  in  reading  were  identified 
and  questionnaires  were  given  to  their  reading  teachers  (whoever  took  primary 
responsibility  for  the  student's  reading  instruction).    All  grade  8  students 
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assessed  in  writing  were  identified  and  questionnaires  were  given  to  their 
writing  teachers. 

The  teacher  questionnaire  included  three  sections  and  took  approximately 
20  jinutes  to  complets.    The  first  section  focused  on  individual  student 
information.    All  teachers  whose  students  had  participated  in  the  assessment 
were  asked  to  respond  to  a  set  of  questions  about  their  students.  Each 
teacher  answered  this  set  of  questions  for  up  to  10  different  students.  Most 
of  the  questions  were  new. 

The  second  section  focused  on  classroom  information.    Teachers  were 
asked  to  respond  to  a  set  of  questions  about  the  classes  in  which  the  students 
in  the  assessment  were  enrolled.     Each  teacher  answered  this  set  of  question 
for  up  to  five  different  classes.    Most  of  the  questions  in  this  section  were 
new  in  1988,  although  some  used  wording  that  is  parallel  to  student  questions 
from  prior  assessments. 

The  third  section  focused  on  teachers'  background  and  exp**^lence. 
Almost  all  of  these  items  were  taken  from  prior  assessmer*"3 . 

The  school  characteristics  and  policies  questionnaire  was  given  to  the 
principal  in  each  participating  school,  and  took  about  11'  minutes  to  complete. 
Three  versions  were  administered — one  for  each  of  the  three  grade/age  levels. 
The  questions  asked  about  the  principal's  background  and  experience,  school 
policie*'.  programs,  and  facilities,  and  the  composition  and  background  of  the 
student  body. 

The  excluded  student  questionnaire  was  given  to  the  teachers  of  students 
who  were  identified  in  the  NAEP  sample  but  were  determined  by  the  school  to  be 
unable  to  participate  because  they  vere  milcly  mentally  retarded  (educable), 
functionally  disabled,  or  had  limited  Englisii  proficiency.     This  questionnaire 
took  approximately  three  minutes  per  student  to  complete  and  asked  about  the 
nature  of  the  student's  exclusion  and  special  programs  in  which  the  student 
participated. 


2.8  FIELD  TESTS  FOR  THE  1988  ASSESSMENT 

By  February  1987,  staff  had  securcid  school  cooperation  for  participation 
in  field  testing  for  the  1988  assessments  of  reading,  writing,  Ci^vics,  and 
U.S.  history  and  for  the  teacher  and  school  policies  questionnaires.     By  this 
time,  staff  had  assembled  84  15-minute  blocks  of  reading,  writing,  civics,  and 
U.S.  history  cognitive  items  and  combined  these  into  34  field  test 
booklets — 13  booklets  at  grade  4,  11  booklets  at  grade  8,  and  10  booklets  at 
grade  12.    Upon  receipt  of  clearance  for  the  field  testing,  booklets  were 
printed  and  approximately  30  trained  administrators  imn.ediately  began  field 
testing.     The  field  tests,  involving  6,800  students  in  "2  school  distr^cf^ 
across  the  country,  were  conducted  in  February  1987.     ETS  staff  members 
traveled  to  12  districts  in  the  Northeast,  17  in  the  Southeast,  1?  in  the 
Central,  and  11  in  the  West.    Generally,  field  tests  were  conducted  at  al"* 
three  levels  in  each  school  district  including  two  classes  at  fourth  grade, 
and  one  class  at  each  grades  8  and  12.     Field  tei^ts  were  carried  out  in  104 
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fourth-grade  classes,  88  eighth- grade  classes,  and  80  twelfth- grade  classes. 
Approximately  300  responses  were  obtained  to  each  question  in  the  field  test 
materials. 

Overall  the  field  test  was  completed  on  schedule.  The  data  were 
collected,  scored,  and  analyzed  in  preparation  for  meetings  with  the  Learning 
Area  Committees  and  Background  Advisory  Committee.    Using  item  analysis,  wh'^h 
provides  the  mean  percentage  of  correct  responses  for  each  item  in  the  field 
test,  committee  members  and  MEP  staff  reviewed  the  materials  according  to 
five  purposes:  to  determine  which  items  were  most  related  to  achievement  in 
the  four  subject  areas;   *:o  evaluate  the  effectiveness  of  items  designed 
specifically  to  assess  higher-order  thinking  skills;  to  dete^^'^ine  necessary 
revisions  to  items  that  lacked  clarity,  or  to  ineffective  item  formats;  to 
give  priority  to  items  to  be  included  in  the  full  assessment;  and  to  determine 
appropriate  timing  for  assessment  items. 

The  geography  background  questionnaire  and  the  blocks  of  cognitive  items 
were  field  tested  in  April  1987.  Trained  administrators  conducted  field  tests 
at  the  twelfth  grade  at  about  12  locations  around  the  country. 

Once  the  committees  had  selected  the  1988  assessment  items,  all  items 
were  rechecked  for  content,  measurement,  and  sensitivity  concerns.  The  0MB 
clearance  process  was  initiated  April  28,  1987  with  the  submission  of  draft 
materials  to  the  Office  of  Educational  Research  and  Improvement.  The  final 
package  was  submitted  May  27,  1987.  Throughout  the  clearance  process  revision 
were  made  In  accordance  vlth  changes  required  by  OERI,  the  IMCD,  and  0MB. 


2.9      FINAL  PREPARATION  OF  THE  1988  ASSESSMENT  MATERIALS 
2.9.1  Objectives  Booklets 

The  objectives  booklets  were  sent  out  for  external  review  by  teachers, 
educators,  and  state  representatives.    The  many  good  suggestions  generated 
from  these  reviews  were  used  to  revise  the  booklets.    After  consensus  review 
and  approval  by  each  Learning  Area  Committee,  the  objective  booklets  were  sent 
to  OERI  for  the  Internal  government  and  peer  review  process.    Subsequent  to 
that  30-day  review  and  Incorporation  of  suggested  revisions,  the  booklets  were 
submitted  to  ETS  Internal  editing  and  finally  to  the  composition  and  printing 
process      The  new  objectives  booklets  for  reading,  writing,  civics,  and  U.S. 
history  vere  published  and  released  In  1987;  the  geography  objectives  booklet 
was  published  and  released  In  1988. 


2.9.2  Student  Assessment  Booklets  and  Questionnaires 

The  Items  earmarked  by  each  Learning  Area  Committee  and  submitted  to  0MB 
clearance  for  Inclusion  In  the  1988  assessment  were  assembled  Into  blocks  in 
accordance  with  the  assessment  design.    These  blocks  were  assembled  to  meet 
content,  context,  cognition,  and  psychometric  specifications  and  to  contorm  to 
the  assessment  time  and  administration  restrictions.    A  common  core  of 
questions  about  students'  demographics  and  home  backgrounds  appeared  at  the 
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beginning  of  each  booklet.    Approximately  five  minutes  of  background  questions 
related  to  a  subject  area  appeared  next,  followed  by  blocks  of  cognitive  items 
in  the  same  subject  area.    For  reading,  seven  blocks  of  cognitive  items  were 
prepared  for  each  grade/age  level;  for  writing,  10  blocks  of  cognitive  items 
were  prepared  for  each  grade/age  level;  for  civics,  three  cognitive  item 
blocks  were  prepared  for  grade  4/age  9  and  10  for  each  of  the  two  older 
grade/age  levels;  for  U.S.  history,  ^hree  blocks  of  cognitive  items  were 
prepared  for  grade  4/age  9  and  .seven  for  each  of  thv.  two  older  grade/age 
levels;  and  for  geography,  three  blocks  of  cogniti''  .  items  were  prepared  for 
grade  12/age  17.     The  questions  for  school  administrators,  teachers,  and  about 
excliAed  students  were  assembled  into  questionnaires. 

In  addition  to  the  main  assessment    special  studies  were  included  to 
measure  the  relationship  between  length  of  time  peraitted  foi  writing  and 
writing  performance  and  the  relatiorship  between  document  literacy  and  reading 
proficiency.     Further  details  about  assessment  instruments  appear  in 
Chapter  4. 

Subsequent  to  assembly,  all  of  the  assessment  blocks  and  questionnaires 
were  subjected  to  the  review  process  established  by  ETS  and  NA.EP,  including 
scrutiny  by  subject-matter  specialists,  measurement  specialists,  test  editors, 
and  persons  specially  trained  to  review  questions  for  any  potential 
insensitivity  to  women  or  minority  groups.    As  part  of  the  0MB  clearance 
process,  all  items  were  also  reviewed  by  OERI  staff,  by  the  Office  for  Quality 
Assurance,  and  by  the  0MB.     Subsequent  to  final  0MB  clearance  on  August  11, 
1987,  the  blocks  atiJ  questionnaires  were  submitted  for  composition,  printing, 
bundling,  and  distribution  to  the  Wesuat,  Iiic,  field  staff  responsible  for 
administering  the  1988  assessment. 
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Chapter  3 
SAMPLE  DESIGH 


Keith  F.  Rust  and  Morris  H.  Hansen 
Westat,  Inc. 


The  samples  for  the  1988  MEP  assessment  were  selected  using  a  complex 
multistage  sample  d^^sign  involving  the  sampling  of  students  from  selected 
schools  w.lthin  94  selected  geographic  areas,  called  primary  sa'^pling  units 
(PSUs),  £.cross  the  United  States. 

The  sample  design  had  four  stages  of  selection: 

1)  selection  of  geographic  PSUs  (counties  or  groups  of  counties) ; 

2)  selection  of  schools  within  PSUs; 

3)  assignment  of  session  types  to  schools;  and 

4/        selection  of  students  for  session  types  within  schools. 

The  samples  were  drawn  for  the  three  different  age  classes,  and  for  each 
age  class  thfe  samples  were  of  two  distinct  types.    The  fir,  1  type  consisted  of 
the  cross -sectional  or  "main"  samples,  while  the  second  type  consisted  of  the 
trend  or  "bridge"  samples.    The  populations  surveyed  with  each  of  these  sample 
types  are  defined  in  Table  3-1.    Separate  samples  of  schools  were  required  for 
the  bridge  samples  and  main  samples  >  because  of  various  differences  in  the 
calendar  period  for  test  administration,  the  format  of  the  administration, 
and,  in  the  case  of  age  class  17  >  the  grade  and  age  definition  of  the 
population  of  interest. 

Table  3-1 

Reference  Populations  for  the  Components  of 
the  NAEP  1988  Samples 

Bridge  to  198A 
and 

Age  Class     Main  Samples  Bridge  to  1986  Civics  Bridge 

9  Sorn  1978  «i)d/or  enrol' od  Born  1978  rnd/or  •nrollod  Bom  1978 

in  sradA  h  in  sr«d«  h 

13  Bom  197^  and/or  •nrollrd  Bom  197A  and/or  anrollad  B^^m  1974 

in  srade  8  in  srada  8 

♦/  Bom  1970  end/or  enrollad  Bom  10/70-9/71  and/or  Bojn  10/70-9/71 

in  srade  12  enroUed  in  srada  11 
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In  addition  to  representing  the  respective  populations  as  a  whole,  ihe 
main  samples  had  as  a  component  a  modest  oversampling  of  schools  vith  moderate 
or  high  enrollment  of  Black  and/or  Hispanic  students  (see  section  3.2).  This 
oversampling  was  undertaken  to  increase  the  sample  s   :es  of  such  students, 
thus  increasing  somewhat  the  reliability  of  estimates  for  these  minority 
groups . 

The  overall  assessment  period  fell  into  three  time  periods  -  fai.1, 
winter,  and  spring.    Not  all  assessment  components,  were  conducted  in  each  time 
perio^l.    Table  3-2  shows  the  relationship  between  the  various  sample 
components  and  the  assessnent  periods.    The  sizes  of  the  PSU  and  school 
samples  and  the  procedures  for  their  selection  v^ere  determined  by  the 
iissessment  period^  as  well  as  by  the  population  to  be  surveyed  and  the  method 
of  administration  in  each  case. 

Table  3-2 

Assessment  Type  by  Age  Class  and  Assessment  Period 

Fall  Winter  Spring 

Age  Class      10/12/87-12/18/87  1/4/88-3/11/88  3/14/88-5/1-^/88 

9  -  Croa3"a9ctional  (part)  Cross-sacclonal  (part) 

Brldsfls  to  l2Bk,  1986 

13  BrldRCs  to  1984    1986,  Cross-soctlonal  (part)  Crosa-a^ictlonal  (part) 

Civics  Brldsa 

17  -  Croas-soctlonal  (part)  Croat- factional  (part) 

Brldses  to  1984.  1986. 
Civics  Brldfa 

This  chaDte.  gives  details  of  the  sample  selection  procedure,  and 
information  on  the  results  of  the  sampling  process.     Still  fuller  details  are 
given  in  The  1988  National  Assessment  of  Educational  Progress — Sampling  and 
Weighting  Procedures f  F^nal  Report  (Rust,  bethel,  Burke,  &  Hansen,  1990). 


3.1      PRIMARY  SAMPLING  UNITS 

In  the  first  sta£;e  of  sampling,  the  United  ;>tates  (the  50  states  FAxd  the 
District  of  Columbia.'  wcs  divided  into  geographic  primary  sampling  units 
Z'PSUs).    Each  PSU  met  a  minimum  size  requirement  (a  population  of  at  lea;: 
60,000  in  the  1980  Census)  and  comprised  either  a  metropolitan  statistical 
area  (MSA),  a  singlf   county,  or  (more  usually  in  the  case  of  nonMSA  PSUs)  a 
group  of  contiguous  ujunties.     In  the  case  of  New  England  MSAs,  which  are  not 
formed  from  whole  counties,  the  correspond' ..g  New  England  County  Metropolitar 
Areas,  which  are  defined  in  terras  of  whole  counties,  were  Jesignared  as  ^he 
PSUs.    The  Now  York  City  MSA  was  d'^ided  along  conty/borough  lir.es  inco  rhiee 
PSUs  for  reasons  of  administrative  ai. '  sampling  convenience.     Each  PSU  wsi> 
contained  entirely  within  one  of  the  four  regions  defined  in  Tab^e  3  3.  These 
regions  were  used  to  stratify  the  sample  of  PSUs,  ensuring  that  each  region 
was  adequately  represented  in  the  various  assessment  samples. 
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Trble  3-3 

Geographic  Regions  Used  for  Stratification 


Northeast 


Southeast 


Central 


West 


Connecticut 
Delaware 
District  of 


Columbia 
Maine 
Maryland 
Massachusetts 
New  Hampshire 
New  Jersey 
New  York 
Pennsylvania 
Rhode  Island 
Vermont 


Alabama 

Arkansas 

Florida 

Georgia 

Kentucky 

Louisiana 

Mississippi 

North  Carolina 

South  Carolina 

Tennessee 

Virginia* 

West  Virginia 


Kansas 


Michigan 
Minnesota 
Missouri 
Nebraska 


North  Dakota 
Ohio 

South  Dakota 
Wisconsin 


Illinois 

Indiana 

Iowa 


Al aska 

Arizona 

Ca'tifornia 

Colorad>^ 

Hawaii 

Idaho 

Montana 

Nevada 

New  Mexico 

Oklahoma 

Oregon 

Texas 

Utah 


Virginia* 


Washington 
Wyoming 


*  That  part  of  Virginia  that  is  included  In  the  Washington,  DC.  metropolitan  statistical 
area  is  included  in  the  Hortheast  region,  the  remainder  of  the  state  being  included  m  the  Southeast 
region. 


In  a  few  cases  an  MSA  crossed  region  boundaries.     Such  MSAs  were  split 
into  two  or  more  PSUs  as  necessary  (e.g.,  the  Cincinnati  OH-KY-IN  MSA  was 
split  into  the  Cincinnati  OH-IN  PSU  in  Region  3  and  the  Cincinnati  KY  PSU  in 
Region  2).    TVelve  subuniverses  of  PSUs  were  then  defined  as  described  below. 

The  28  largest  PSUs  were  included  in  the  sample  with  certainty.  An 
additional  six  very  large  PSUs  (four  from  the  Southeast  and  two  from  the  West) 
that  had  large  proportions  of  Black  students  and/or  Hispanic  students  were 
also  included  with  certainty.    The  34  certainty  PSUs  constituted  32 
metropolitan  areas,  since  the  New  York  City  MSA  was  divided  into  three 
certainty  PSUs.    The  inclusion  of  these  34  PSUs  in  the  sample  with  certainty 
provided  an  approximately  optimum  cost-efficient  sample  of  schools  and 
students  when  samples  were  drawn  within  them       tht-  required  national  sampling 
rate.    The  representativeness  of  the  sample  for  minority  groups  was  enhanced 
by  ensur  ig  that  these  PSUs  were  included  in  the  sample,  since  these  minority 
groups  are  relatively  heavily  represented  within  these  certainty  PSUs.  The 
remaining  smaller  PSUs  were  not  guaranteed  to  be  selected  for  the  sample. 
These  were  grouped  into  a  number  of  noncertainty  strata  (so  called  because  tht 
PSUs  in  these  strata  were  not  included  in  the  sample  vith  certainty),  and  a 
sample  PSU  was  selected  from  each  stratum. 

The  PSUs  were  classified  into  four  regions,  each  contdJning  about  one 
fourth  of  the  U.S.  population.    These  regions  were  defined  primarily  by  state 
(Table  3-3).    In  each  region,  noncertainty  PSUs  were  cla.jsified  as  MSA  or 
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nonMSA.    In  the  Southeast  and  West  regions,  the  PSUs  in  which  the  combined 
proportion  of  population  which  were  Black  and  Hispanic  respectively  in  the 
1980  Census  exceeded  20  percent,  were  classified  as  high  rainori::y.  The 
resulting  major  strata,  or  subuniverses,  are  shown  in  Table  3-4. 

Table  3-4 
The  Sampling  Subuniverses 
and  the  Nximber  of  Noncertainty  Strata  in  Each 

  MSA  PSUs      NonMSA  PSUs   

Regular    High-minority  Regular  High-minority 


Region 

Strata 

Strata 

Strata 

Strata 

Northeast 

8 

2 

Southeast 

4 

6 

4 

6 

Central 

8 

6 

West 

4 

6 

4 

2 

Total 

24 

12 

16 

8 

Within  each  major  stratum  (subuniverse) ,  further  stratification  was 
achieved  by  ordering  the  noncertainty  PSUs  according  to  several  additional 
socioeconomic  characteristics,  yielding  60  strata.    The  strata  were  defineu 
that  the  aggregate  of  the  measures  of  size  of  the  PSUs  in  a  stratum  was 
approximately  equal  for  each  stratum,  except  for  strata  in  the  high- minority 
subuniverses,  in  which  the  aggx.egate  was  approximately  half  that  of  the 
regular  strata.     The  size  measure  used  was  the  population  from  the  1980 
Census-    The  characteristics  used  to  define  strata  were  the  percent  minority 
population,  the  percentage  change  in  total  population  since  1970,  the  per 
capita  educational  expenditure,  and  the  percent  of  persons  employed  in 
manufacturing  (MSA  subuniverses  only)  and  tlie  percentages  of  rural  and  urban 
dwellers  (nonMSA  subuni'/erses  only).     One  PSU  was  selected  with  probability 
proportional  to  size  from  each  of  the  60  noncertainty  strata.    That  is,  within 
each  stratum,  a  PSU's  probability  of  being  the  sample  selection  from  that 
stratum  was  proportional  to  its  population.    Thus  the  high-minority 
subuniverses  were  sampled  at  approximately  twice  the  rate  of  the  other 
subuniverses,  since  they  were  about  half  as  large.     This  procedure  of 
oversampling  from  the  high-minority  subuniverses  was  used  with  the  aim  of 
reducing  somewhat  the  level  of  sampling  error  for  estimates  relating  to  the 
populations  of  Black  and  Hispanic  students. 

The  final  sample  of  94  PSUs  was  drawn  from  a  population  of  about  1,000 
PSUs.     Primarily  because  of  the  use  of  MSAs  as  PSUs,  PSUs  varied  considerably 
as  to  their  probability  of  selection,  since  they  varied  greatly  in  size.  The 
34  certainty  PSUs  consisted  of  the  26  largest  MSAs  in  the  country,  based  on 
the  1980  population  from  the  Census,  plus  six  other  large  MSAs  from  the 
Southeast  and  West  regions  with  in  excess  of  20  percent  of  their  population^ 
being  Black  or  Hispanic.    The  36  selected  noncertainty  MSA  PSUs  had 
probabilities  of  selection  ranging  from  0.028  to  0.584,  while  the  24  selected 
nonJiSA  PSUs  had  probabilities  ranging  from  0.021  to  0.101.    The  variations  in 
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probability  depended  upon  the  size  of  the  PSU  (1980  population)  and  whether  or 
not  the  PSU  was  in  a  high  minority  subuniverse.    Parts  of  37  states  were 
included  in  the  94  selected  ^  Us. 

The  PSU  samples  were  drawn  at  one  time  for  the  1986,  1988,  1990,  .^nd 
1992  assessments.    They  were  drawn  so  as  to  provide  for  the  rotation  of  the 
PSUs  from  one  assessment  to  the  next,  except  that  certainty  PSUs  were  retained 
in  each  assessment  year,  and  some  of  the  larger  noncertainty  PSUs  were 
retained  for  two  successive  assessment  years. 

The  94  PSUs  were  used  for  the  main  assessments  of  all  three  age  classes. 
To  facilitate  and  improve  administration  of  the  assessments  and  co  allow  for 
the  estimation  of  growth  in  achievement  during  the  school  year,  the  assessmei^L 
sample  was  divided  into  ti^     randomly  equivalent  subsamples,  une  to  be  asst  sed 
in  the  winter  and  the  other  to  be  assessed  in  the  spring.     For  this  purpose, 
the  94  PSUs  were  designated  as  winter  PSUs,  spring  PSUs,  or  both  winter  and 
spring  PSUs,  according  to  the  following  scheme.    The  18  largest  certainty  PSUs 
were  designated  both  winter  and  spring  PSUs,  to  be  included  in  the  sample  for 
both  seasons  (the  sample  of  schools  within  each  of  these  PSUs  was  divided  into 
two  stratified  random  half -samples;     one  half-sample  to  be  assessed  in  the 
winter  and  one  to  be  assessed  in  the  spring) .     The  16  smaller  certainty  PSUs 
were  ordered  by  region  and  then  alternately  designated  as  winter  PSUs  or 
spring  PSUs,  resulting  in  eight  PSUs  for  each  season.     Similarly,  alternate 
members  of  the  set  of  the  60  noncertainty  PSUs,  arranged  in  stratum  order 
within  each  subuniverse,  were  designated  as  winter  or  spring  PSUs.    The  end 
result  was  56  winter  PSUs,  including  38  in  which  assessments  were  conducted 
only  during  the  winter,  and  18  where  assessments  were  conducted  in  the  winter 
and  the  spring,  and  56  spring  PSUs,  consisting  of  38  in  which  assessments  were 
conducted  only  in  the  spring,  and  the  18  winter  and  spring  PSMs. 

The  procedure  of  designating  two  half- samples  for  the  main  assessment, 
one  for  winter  and  one  for  spring,  differed  from  the  procedure  used  in  1986. 
In  that  year,  all  of  the  main  (i.e.,  cross -sectional)  sample  assessments  took 
place  in  the  spring  within  ^he  94  selected  PSUs;  consequently,  it  was  not 
necessary  to  designate  half -samples  on  th    basis  of  season  of  assessment. 

The  bridge  assessments  used  a  series  of  subsamples  of  the  94  PSUs  used 
for  the  main  assessment.    The  age  class  9  bridge  assessments,  whi^^n  were 
conducted  in  the  winter,  used  the  56  PSUs  designated  as  winter  PSUs  in  the 
main  assessment;  the  age  class  17  bridge  assessments,  conducted  in  the  spring, 
used  the  56  PSUs  designated  as  spring  PSUs.    The  age  class  13  bridge 
assessments,  conducted  in  the  fall,  used  64  PSUj  selected  from  the  complete 
set  of  94  PSUs  witt  probability  proportional  to  the  measure  of  size  of  the 
strata  from  which  the  PSUs  were  selected.    The  selection  of  PSUs  for  the  fall 
assessTient  was  independeiit  of  the  pro.:ess  of  designating  PSUs  as  winter  or 
spring.     As  for  the  winter  and  spring  subsamples,  the  18  largest  certainty 
PSUs  were  retained  in  the  fall  bridge  sample  with  certainty.     The  distribution 
of  PSUs  among  the  fall,  winter,  and  spring  assessment  periods  is  illustrated 
in  Figure  3-1 
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Figure  3-1 

Distribution  of  94  Sampled  PSUs  by  Inclusion 
in  the  Fall,  Winter,  and  Spring  Assessment  Periods 


8  smaller 

30  noncertainty  Qortainty 

PSUs  PSUn      I  Spring  samplo  (56  PSUs) 


.18 

6 

18  largest 
certainty 
PSUs 

2 

20 

2 

6 

12 

10 

Winter  sample  (56  PSUs)   -\  8  smaller  30  noncartainty 

certainty  PSUs 
PSUs 


Denotes  fall  sample  (64  PSUs) 


The  age  class  9  and  age  class  13  bridge  samples  used  the  same  school  and 
student  eligibility  requirements  as  the  respective  main  samples. 
Nevertheless,  special  bridge  samples  were  required  because 

1)  The  conditions  for  administration  of  the  arsessment  varied 
considerably  between  the  main  sample  and  bridge  sample  sessions. 

2)  The  need  in  the  bridge  samples  for  four  distinct  session  types  for 
age  class  9  and  five  for  age  class  13  made  it  infeasible  to 
conduct  both  a  main  sample  session  and  the  several  bridge  sessions 
within  a  given  school. 

3)  For  age  class  13,  the  main  samples  were  conducted  at  an 
inappropriate  time  of  the  year  for  bridging  purposes,  so  that  a 
sample  of  schools  was  needed  to  undertake  the  bridge  assessments 
in  the  fall  of  1987. 


A  separate  sample  of  schools  was  required  for  the  bridge  sessions  and 
the  main  sessions  for  age  class  17  primarily  because  the  definitions  for 
student  eligibility,  based  on  age  and  grade,  differed  substantially  becw?en 
the  *  o  samples,  even  though  the  same  population  of  schools  was  surveyed  in 
each  case.     Conlitions  of  administration  varied  somewhat  also. 
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3.2      SCHOOLS  FOR  MAIH  SAMPLES 


In  the  second  stage  of  sampling,  the  public,  private.  Catholic,  Bureau 
of  Indian  Affairs,  and  Department  of  Defense  schools  within  each  of  the  94 
PSUs  were  listed  according  to  the  grade  ranges  associated  with  the  three  age 
classes.    Tlie  population  of  eligible  schools  for  each  age  class  (Table  3-5) 
was  the  same  for  bridge  anO  main  samples  in  each  case.    Any  school  having  one 
or  more  of  the  eligible  grades,  and  located  within  an  appropriate  PSU,  was 
included  on  the  frame  of  schools  (the  list  of  schools  from  which  the  samples 
of  schools  were  drawn)  for  a  g^/en  sample.     For  each  age  class,  only  a 
fractio  i  of  one  percent  of  age -eligible  students  was  enrolled  in  ineligible 
schools.     Each  school  within  the  94  PSUs  with  a  grade  in  the  range  of  2  to  12 
was  included  in  at  least  one  age  class — a  total  of  36,290  schools.  An 
independent  sample  of  schools  was  selected  for  each  of  the  age  classes.  Thus 
some  schools  were  selected  for  assessment  of  two  age  classes,  and  a  few  were 
selected  for  all  three.    The  lists  of  schools  were  obtained  from  the  1986  list 
of  schools  maintained  by  Quality  Education  Data,  Inc. 


Table  3-5 

Grade  Definition  of  School  Eligibility  for  Frame  Inclusion 
and  Frame  Sizes,  Main  and  Bridge  Samples 

Frame  included  schools 

Age  Class                 with  an^^  grade  in  the  rangp  Number  of  schools  on  frame 

9                                          ?-5  26,951 

13                                         6-9  28,167 

17                                          9-12  8,485 

Total                                     2-12  36,290 


Schools  within  each  PSU  were  selected  (without  leplacement)  with 
probabilities  proportional  to  assigned         ures  of  size.    Roughly  equal 
measures  of  size  were  assigned  to  schools  containing  estimates  of  grade/age - 
eligible  students  ranging  from  20  to  150  (for  age  class  9),  or  to  200  (for  age 
class  13  and  age  clar,s  17).    Schools  larger  than  the  indicated  maximum  s^ze 
were  selected  within  the  selected  PSUs  with  probabilities  proportional  to  the 
number  of  grade/age-eligible  students.    As  a  result,  in  some  instances  a  large 
school  was  selected  with  certainty  within  a  relatively  small  selected  PSU. 
Schools  with  fewer  than  20  estimated  grade /a^e  eligibles  were  assigned 
considerably  lower  measures  of  size,  and  thus  lower  probabilities  of 
selection,  since  assessment  in  these  schools  Involved  substantially  higher 
per-student  administrative  co:  ts. 

A  school  with  minority  (Black  and/or  Hispanic)   enrollment  in  excess  2 
15  percent  of  total  enrollment  was  given  double  the  probability  of  selection 
of  a  school  of  similar  size  in  the  same  PSU  with  minority  enrollment  below  15 
percent.    Overall  probabilities  of  selection  for  such  high-minority  schools 
were  twice  those  for  other  schools  of  the  same  size  from  a  given  PSU  In  order 
to  enlarge  the  sample  of  Black  and  Hispanic  students,  thereby  enhancing  the 
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reliability  of  estimates  for  these  groups.    For  a  given  overall  size  of 
sample,  this  procedure  reduces  somewhat  the  reliability  of  estimates  for  all 
students  as  a  whole  and  for  those  not  Black  or  Hispanic. 

The  total  number  of  schools  selected  for  each  age  class  was  determined 
to  be  such  that  the  predesignated  student  sample  sizes  would  be  achieved  by 
selecting  all  eligible  students  in  a  selected  school,  up  to  a  maximum  of  150 
(for  age  class  9)  or  200  (for  age  class  13  and  age  class  17),  allowing  for 
losses  due  to  nonparticipation  of  selected  schools  and  students  and  the 
exclusion  of  students  from  the  ass  ssment. 

In  each  of  the  94  PSUs,  a  minimum  of  three  schools  was  selected  for  age 
class  9,  four  schools  for  age  class  13,  and  two  schools  for  age  class  17. 
These  minima  were  established  based  on  the  total  number  of  students  and  hence 
schools  required  for  the  particular  age  class,  and  on  the  proportion  of 
selected  schools  likely  to  contain  few  or  no  eligible  students.  This 
proportion  is  much  higher  for  age  class  13  than  for  the  other  age  classes 
because  of  the  inclusion  on  the  frame  for  this  age  class  of  commonly  occurring 
elementary  schools  with  a  grade  span  of  kindergarten  through  grade  6,  and  high 
schools  with  grade  9  through  grade  12,  which  generally  enroll  few  13 -year- 
olds.     The  use  of  these  minima  for  the  sample  size  of  schools  per  PSU  was 
derived  as  a  compromise  between  two  desirable  but  conflicting  objectives.  The 
first  of  these  is  to  ensure  substantial  representation  from  within  each 
selected  "   J  (after  the  impact  of  nonresponse) .    The  second  objective  is  the 
need  to  keep  the  variability  in  overall  student  sampling  probabilities  (and 
hence  weights)  to  a  low  level,  so  as  to  control  the  sampling  errors  associated 
with  ^-EP  estimates.     The  use  of  a  large  minimum  sample  size  requirement  per 
PSU  would  act  to  reduce  unduly  the  weights  of  studerts  selected  from  small 


This  design,  w:th  the  important  exceptions  described  above,  had  the  goal 
of  yielding  a  sample  of  students  in  a  given  age  or  grade  with  approximately 
uniform  probabilities  of  selection.    The  practical  constraints  on  the  sample 
size  within  each  school,  and  the  need  to  ensure  an  adequate  ..ample  within  each 
PSU,  resulted  in  some  substantial  violations  of  this  general  goal. 

For  all  three  age  classes,  a  sample  of  schools  was  fir      drawn  for  the 
bridf<»  assessments  (see  below).    These  schools  were  then  excl^^^ed  from  the 
frame  when  the  samples  of  schools  were  drawn  for  the  main  assessments. 
Adjui        ts  were  made  to  reflect  the  appropriate  probabilities  of  selection, 
and  tii^^  wlie  sample  weights,  to  yield  unbiased  estimates  for  both  bridge  and 
main  samples.     Schools  '•elected  for  main  assessments  were  further  classified 
as  belonging  to  the  winter  main  assessment  or  the  spring  main  assessment.  For 
the  schools  in  the  PSUs  assigned  to  a  single  season,  this  classification 
corresponded  to  the  classification  of  the  PSU  containing  the  school.  Half- 
samples  of  the  schools  in  each  of  the  18  largest  certainty  PSUs  (those  PSUs 
assigned  to  both  seasons)  were  assigned  to  the  winter  and  spring  assessments 
respectively. 

After  selection  of  the  initial  sample  of  schools  was  completed, 
information  was  obtained  to  update  the  sample  foi.  new  eligible  schools. 
Public  school  districts  and  Catholic  dioceses  of  initially  selected  schools 


PSUs. 


58 


were  asked  to  give  informatior  about  new  schools  and  schools  wich  changes  in 
grade  structure  since  tha  1986  date  tc  which  the  Quality  Education  Data  llsc 
related,  for  their  district/diocese.    Schools  so  identified  were  given  an 
appropriate  chance  of  inclusion  in  each  of  the  samples  for  which  they  were 
eligible.    The  overall  probability  of  inclusion  for  a  given  age  class  foi  each 

such  school  was  determined  by  tho  estimated  number  of  eligible  Si  ^ents 

enrolled  in  the  school,  and  the  within- PSU  sampling  rate  used  to  select  the 
initial  sample  of  schools.    The  conditional  probability  of  selection,  used  to 
draw  the  actual  samples  of  new  schools,  was  obtained  by  dividing  the  overall 
probability  for  the  school  by  the  probability  that  the  school  district  was 
represented  by  a  selected  school  in  one  of  three  initial  main  samples.  These 
district  level  probabilities  were  obtained  by  reconstructing  the  sample 
selection  procedure  to  obtain  the  probability  that  at  least  one  school  from 
the  district  was  included  in  a  given  age  class  sample,  and  then  combining 
these  independent  probabilities  across  age  classes.    The  process  identified  a 
total  of  58  new  schools,  44  eligible  for  age  class  9,  40  eligible  for  age 
class  13,  and  11  eligible  for  age  class  17.    Three  new  schools  were  added  to 
the  sample       this  way — two  at  age  class  9  and  one  at  age  class  13.    All  three 
schools  were  found  to  have  eligible  students  enrolled,  and  were  invited  to 
participate . 

In  a  few  PSUs  where  school  refusals  were  relatively  heavy  for  a 
particular  sample,  substitute  school  selections  v^ere  made,  replacing  the 
refusals  (to  the  extent  feasible)  with  schools  from  within  the  same  PSU  and 
similar  in  size,  affiliation  (public.  Catholic,  or  other  private),  grade  span, 
and  minority  composition.     In  a  very  few  cases,  substitute  schools  were  choseii 
from  another  PSU  within  the  same  stratum  as  the  original.     The  goal  of  this 
procedure  was  to  maintain  the  student  sample  sizes  needed,  while  keeping 
variance  and  nonresponse  bias  at  acceptable  levels.    Table  3-6  shov;s  the 
number  of  in-scope  schools  selected,  cooperating,  and  substituted,  in  each  of 
the  school  samples.     The  participation  rates  given  are  based  on  the  initially 
selected  sam^.  le  of  schools.    These  response  rates  are  comparable  with  those  of 
previous  assessments  conducted  during  the  1980s.    Note  that  since  the  response 
rates  quoted  do  not  include  the  substitute  selections,   the  potential  for 
nonresponse  bias  is  likely  to  be  somewhat  less  than  these  rates  would 
indicate.    This  is  because  the  substitute  selections  were  chosen  based  on 
their  similarity  to  the  initially  refusing  selections. 

The  considerable  numbers  of  schools  selected  with  no  eligible  students 
enrolled  resulted  primarily  from  the  fact  that,  for  example,  for  ^rade  8/age 
13,  some  schools  with  grades  6,  7,  or  9,  but  no  grade  8,  were  sampled.  Such 
schools  had  a  reasonable  chance  of  containing  some  age  13  students.     Of tea 
they  did  have  a  number  of  eligible  students,  but  sometimes  they  had  none. 
Because  of  the  grade  structure  of  schools,  this  occurred  most  often  for  grade 
8/age  13. 


Table  3-6 

School  Sample  Sizes,  Refusals,  and  Substitutes 
Main  Samples 


A{.'e  Class  9       Age  Class  13       Ape  Class  17  Total 


Selected,  in  scope  373 

Refusals  42 

Participation  rate  of 

originally  selected  schools  89% 

1986  participation  rate  893i;* 

Participating;  no  eligible 

enrolled  A 

Substitutes  participating  9 

Final  assessed  sample  336 


506 
68 


87X 
89X* 

39 
14 
413 


383 
66 


83X 
81Z** 

13 
8 

312 


1,262 
176 


8f. 
8/x 

56 
31 
1,061 


^    Tho  rates  quoted  £or  1986  are  for  the  srade  S/age  9  and  grade  7/a8e  13  spring  sanples 
respectively. 

No  assessment  for  grade  12/ase  17  was  conducted  before  1988.    Tl-.«  rate  quoted  here  is  for  the 
1986  ^rtCe  ll/age  17  spring  sample. 


A  school  characteristics  and  policies  questionnaire  was  mailed  to  every 
sampled  school  by  Westat  before  the  assessment.     The  Westat  supervisor  then 
collecced  the  questionnaires  and  returned  them  to  ETS.    The  school 
characteristics  and  policies  questionnaire  is  described  in  Chapter  4. 


A  school  principal's  questionnaire,  distributed  to  each  sampled  school 
by  Westat  before  the  assessment,  was  used  to  ref^.ne  the  estimate  of  the 
grade/age- eligible  students  and  to  determine  in  part  the  size  and  type  of 
coramunity  (STOC)  codes  (see  Appendix  G  and  Rust  et  al.,  1990). 


3.3      SELECTION  OF  SCHOOLS  FOR  BRIDGE  SAMPLES;  THE  ASSIGNMENT  OF 
SESSIONS  TO  SCHOOLS 

Schools  were  selected  for  age  class  13  bridge  assessments  (conducted  in 
the  fall  of  1587)  from  the  ..ubsample  of  64  PSDs  that  had  been  designated  as 
the  age  < lass  13  bridge  PSUs.    One  school  or  school  cluster  was  selected  in 
each  PSU,  except  that  three  schools  or  school  clusters  weie  selected  from  each 
of  the  two  largest  PSUs  (which  were  substantially  larger  than  the  remainder). 
Initially,  four  types  of  bridge  sessions  were  to  be  adminis  ered,  and  the 
sample  was  selected  so  as  to  meet  this  requirement,  with  a  maximum  of  four 
sessions  to  be  administered  in  a  school,  with  each  session  type  to  be 
administered  in  each  PSU.    How»   ar,  a  fifth  session  type  was  added  too  late  to 
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amend  the  school  sample  (although  the  total  required  sample  size  of  students 
was  not  changed) .    Thus  the  requirement  that  each  session  type  je  administered 
at  least  once  in  each  PSU  was  modified  so  that  each  session  type  would  be 
administered  at  least  once  in  at  least  52  PSUs.    To  avoid  the  possibility  that 
a  particular  bridge  session  might  be  assigned  to  a  single  school  wit',  only  one 
or  ven/  few  eligibles,  bniall  schools  were  grouped  with  other  schools  in  the 
same  PSU  to  form  clusters  of  a  specified  minimum  number  of  eligibles.  Bridge 
sessions  were  then  assigned  within  each  PSU  by  selecting  a  school  or  school 
cluster  with  probability  proportional  to  the  estimated  number  of  age  and  graJe 
eligibles  within  the  school(s). 

Schools  were  solectea  for  the  age  class  9  bridge  assessmsnt^  (conducted 
in  the  wintar  of  1988)  from  the  subsample  of  56  PSUs  designated  for  the  winte. 
assessment.     Four  tjrpes  of  bridge  sessions  were  to  be  administered.  The 
selection  was  such  that  each  of  the  session  types  used  in  the  bridge 
assessments  would  be  administered  in  at  least  one  school  within  each  of  the  56 
PSUs  designated  as  winter  or  a;  winter  and  s^^ring.     Schools  or  school  clusters 
were  sampled  in  the  same  manner  as  for  age  class  13  except  that  two  schools  or 
school  clusters  were  selected  per  PSU,  with  six  schools  or  school  clusters 
drawn  from  each  of  the  two  largest  PSUs. 

In  a  like  manner,  schools  were  selected  for  the  age  class  17  bridge 
assessments  (conducted  in  the  spring  of  1988)  from  the  subsample  of  56  PSUs 
designated  for  the  spring  assessment.    Three  session  types  were  to  be 
administered.    Each  of  the  session  types  was  to  be  administered  within  at 
least  one  school  or  school  cluster  in  each  of  the  56  PSUs.    Two  schools  (or 
i,^,hool  clusters)  were  selected  per  PSU,  with  six  selections  being  made  from 
each  of  the  two  largest  PSUs. 

Substitute  selections  were  made  for  nonparticipating  bridge  sample 
schools  in  certain  PSUs  in  similar  manner  to  the  approach  used  for  the  main 
sample  schools.     In  the  case  of  the  bridge  samples,  however,  no  samples  of  new 
schools  were  selected.     For  the  bridge  samples,  the  smaller  sample  sizes 
involved  meant  that  the  biasing  effect  of  failing  to  represent  such  new 
schools  was  relatively  less,  as  a  component  of  total  error.     For  the  age  class 
13  sample  in  particular,  there  was  insufficient  time  available  between  the 
point  when  districts  were  contacted  and  when  assessments  were  scheduled  (mid- 
October  through  early  December,  1987)  to  ascertain  the  existence  i  new 
schools,  sample  them,  and  obtain  the  participation  of  the  schools. 

Table  3-7  shows  the  school  sample  sizes  and  participation  rates  for  tie 
bridge  samples  for  each  age  class.  School  participation  rates  are  similar  to 
those  seen  in  comparable  samples  in  1986 — the  fall  and  winter  bridges  nnd  e 
grade  11/age  17  spring  main  sample. 
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Table  3-7 

Scl.ool  Sample  Sizes,  Refusals,  and  Substitutes 
Bridge  Samples 


Grade  4/Age  9 
(Winter) 


Selected,  in  scope  180 

Refusals  23 

Participation  rate  of 

originally  selected  schools  87Z 

1986  participation  rate  87Z 

Participating;  no  eligibles 

enrolled  ^ 

Substitutes  participating  2 

Final  assessed  sample  156 


Grade  8/Ags  13    Grade  11/Age  17 


(Fall)  (Spring)  Total 

219  155  554 

16  34  73 

93X  78X  87Z 

84X  8U  83Z 

30  7  40 

8  6  16 

181  120  457 


For  all  three  age  classes,  sessions  were  assigned  to  bridge  sample 
scn^ols  in  the  following  manner.     First,  the  number  of  sessions  per  school  was 
established.    Thi  5  was  the  maximum  number  of  sessions  (up  to  four)  that  could 
be  administered  without  creating  unduly  small  session  sizes  with  few 
eligibles.    Thus,  in  most  bridge  sample  schools,  four  sessions  were  conducted. 
However,  schools  with  fewer  chan  20  eligibles,  for  example,  were  asked  to 
conduct  only  a  single  session. 

The  number  of  session  types  conducted  in  the  assessment  varied  by  age 
class.     Table  3-8  in  the  following  section  shows,  among  other  things,  the 
various  bridge  sample  session  types  conducted  for  each  age  class,  and  the  year 
of  the  corresponding  assessment  to  which  these  session  types  provided  a 
bridge. 

The  assignment  of  sessions  to  schools  maximized  the  number  of  session 
types  conducted  within  each  PSU.    Thus,  to  the  extent  fe.^sible,  session 
assignment  was  delayed  until  after  it  was  determined  that  a  selected  school 
would  participate.     On  a  few  occasions,  a  session  could  not  be  conducted  in  a 
school  that,  at  the  time  of  session  assignment,  was  expected  ^o  participate 
but  subsequently  did  not.    As  a  result,  two  ^ypes  of  school  nonresponse 
adjustment  factors,  denoted  school  and  session,  were  required  for  the  bridge 
samples  (see  Chapter  8). 

This  procedure  was  intended  to  assure  that  each  session  type  was 
assigned  in  each  PSU  at  le^^st  once  for  the  age  class  9  and  age  class  17 
samples.    At  age  class  12    however,  often  a  PSU  was  represented  iti  the  sample 
by  a  single  large  school.     Since  it  was  not  feasible  to  administer  each  of 
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five  different  session  types  in  a  single  school,  not  all  session  types  were 
administered  in  all  64  PSUs,  but  each  session  type  was  administered  in  at 
least  52  PSUs  for  this  age  class. 


3,4     SAMPLING  STUDENTS 

In  the  fourth  stage  of  sampling,  a  consolidated  list  was  prepared  for 
each  school  of  all  grade -eligible  and  age -eligible  st:udents  for  the  age  class 
for  which  the  school  was  selected.    A  systematic  selection  of  eligible 
students  was  made  from  this  list  (unless  all  students  were  to  be  assessed)  to 
provide  the  target  sample  size.    For  bridge  sample  schools  assigned  to  more 
than  a  single  session  type,  students  were  assigned  by  Westat  district 
supervisors  to  print  or  paced- tape  sessions  using  specified  procedures. 
Students  assigned  to  paced-tape  sessions  who  were  not  age-eligible  were 
dropped  from  the  assessment. 

The  maxima  established  for  the  number  of  students  to  be  scheduled  for 
assessment  were  150  for  age  class  9,  220  for  age  class  13,  and  200  for  age 
class  17.    The  limit  of  220  for  age  class  13  was  raised  from  an  initial  limit 
of  200  owing  to  concern  that  the  selected  sample  of  schools  would  not  yield 
sufficient  assessed  students.     Note  that  at  all  three  age  classes  the  Tioraber 
of  students  invited  to  the  assessment  in  larger  schools  (those  with  numbers  of 
enrolled  eligibles  in  excess  of  these  limits)  was  likely  to  be  significantly 
below  these  specified  limits.    This  was  because  not  only  were  excluded 
students  not  invited,  but  those  students  in  the  modal  grade  who  were  not  age 
eligible  but  were  selected  for  assessment  in  the  tape  sessions  (the  majority 
of  the  sample  at  ages  9  and  13)  were  not  invited  to  participate  (unless 
specifically  requested  by  the  school  for  operational  reasons) . 

The  sample  for  students  to  be  selected  in  each  school  was  derived  in  the 
following  manner,  both  for  main  and  for  bridge  samples.    On  the  basis  of  data 
obtained  from  the  principal  questionnaire  (or  the  sample  frame  when  the 
principal  questionnaiie  data  were  not  obtained  in  time)  an  estimate  of  the 
number  of  grade-  plus  a^e-eligible  students  was  established  for  each  school. 
A  Session  Assignr *nt  Form  was  generated  for  each  school,  showing  the  line 
numbers  (described  below)  of  the  students  to  be  selected  (and  in  the  case  of 
the  bridge  samples,  indicating  the  type  of  session  to  be  taken  by  each  s ich 
student).    These  line  numbers  were  generated  using  a  sampling  interval 
designed  to  give  the  appropriate  sample  size  for  each  school.     Thus  the 
overall  sampling  interval  was  1.0  for  schools  in  which  all  eligible  scuderti, 
were  to  be  assessed.    The  appropriate  sampling  interval  was  specified  for 
schools  with  larger  numbers  of  eligible  ^.uudents,  such  as  to  give  the 
appropriate  maximum  sample  size  (described  above  fur  each  age  class)  in  the 
case  that  the  school  had  an  enrollment  of  eligible  students  exactly  equal  to 
that  predicted. 

If  the  Westat  supervisor  found  that,  when  applied  to  the  numbered  list 
of  eligible  students  assembled  in  the  field  for  each  school,  the  line  numbers 
generated  gave  rise  to  a  sample  in  excess  of  120  percent  of  the  appropriate 
maximum  sample  size  limit  specified  above,  he  or  she  called  Westat 's  central 
office.    By  use  of  a  personal  computer,  new  line  numbers  bast  1  on  the  actual 
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number  of  eligible  students  were  generated  and  relayed  to  the  supervisor.  A 
similar  revision  to  the  line  numbers  was  made  in  the  case  of  a  school  with  a 
sampling  interval  In  excess  of  1.0,  and  eligible  enrollment  less  than  80 
percent  of  that  initially  estimated.    In  this  latter  case  the  sample  si"  was 
increased  to  the  appropriate  level.    This  procedur    gave  a  suitable  com^  ^misc 
between  control  over  the  sampling  rate  within  each  school  and  operational 
autonomy  and  flexibility  for  Westat  field  supervisors.    Note  chat  in  all 
cases,  sampling  intervals  were  generated  in  Westat' s  central  office,  and 
stored  for  use  in  sample  weighting.     Supervisors  were  noL  required  t  deri.^ 
or  record  within-school  sampling  rates. 

Table  3-8  shows  the  number  of  students  per  schvool  who  were  assessed  for 
each  session  type.    Note  that,  for  the  various  spiral  samples,  the  number  of 
students  assessed  per  item  per  school  is  quite  low,  even  though  t^.pically 
dozens  of  students  were  assessed  in  total  in  a  particular  school.  s  the 

extent  of  clustering  of  the  sample  is  in  general  quite  modest,  because  most 
bridge  sample  schools  conducted  a  few  different  types  of  sessions  with  a 
moderate  number  of  students  in  each,  and  more  importantly  because  the  use  of 
BIB  spiraling  in  the  main  samples  and  print- administered  bridge  sample 
sessions  greatly  alleviated  the  effects  of  clustering  the  samples  of  students 
within  schools. 


3.5      EXCLUDED  STUDENTS 

Some  students  selected  for  the  sample  were  deemed  unassessable  by  school 
authorities  because  they  had  limited  English  language  proficiency,  were  judged 
as  being  mildly  mentally  retarded  (educable),  or  were  functionally  disabled. 
In  these  cases,  school  staff  completed  an  excluded  student  questionnaire, 
listing  the  reason  for  exclusion  and  providing  some  background  information. 
For  the  excluded  students,  no  distinction  was  made  as  to  the  season  of  the 
year  in  which  their  school  was  assessed,  since  the  timing  was  unimportant  for 
these  students.     For  age  class  9  and  age  class  13,  no  distinction  was  made 
between  <?tuaents  excluded  from  bridge  assessments  and  s^^udents  excluded  fiom 
the  main  assessment,  since  the  same  grade  and  age  eligibility  definitions 
applied  in  each  case.     Conversely,  for  age  class  17,  the  excluded  students 
from  the  bridge  assessments  (with  an  October-September  age  definition  and 
modal  grade  of  11)  were  treated  as  separate  irom  the  excluded  students  in  tht 
main  assessment  (with  e  calendar-year  age  definition  and  modal  grade  of  12). 

For  all  samples,  all  selected  students,  whether  eligible  by  age.  grade, 
or  both,  were  considered  for  possible  exclusion,  on  the  grounds  that  they 
would  not  ^     able  to  participate  meaningfully  in  the  assessment.    All  students 
so  identified  were  included  in  the  sample  of  excluded  students.    This  occurrea 
even  if  the  student,  had  he  or  she  been  assessable,  would  have  subsequently 
been  dropped  from  a  bridge  tape  session  sample  because  of  age  ineligibility. 
This  represented  a  change  from  the  1986  procedure,  where  only  students  who 
would  otherwise  have  been  invited  (i.e.,  met  the  specific  age  or  grade 
requirements  for  the  particular  type  of  session  for  which  they  were  selected) 
were  included  in  the  sample  of  excluded  students.    The  procedure  for  1988 
provided  consistency  in  the  definition  of  the  excluded  i     dent  samples  (all 
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Table  3-8 


Number  of  Students  per  School  for  Each  Session  Type 

No.  of  Students 

per  Session  Type         No.  of  Students 


Bridge 

No.  of 

per 

School 

per  Item 

per  Schoc 

oaiupie 

Session  Tvne 

Mean 

Max, 

Max, 

Age  uiass  V  bridges 

Spiral  booklets 

51- 

5o 

to  1986 

152 

34.1 

81 

5.7 

14 

Tape  booklet  91 

to  1984 

112 

11.4 

39 

11.4 

39 

Tape  oooklet  9/ 

to  1984 

111 

11.2 

37 

11.2 

37 

Tape  booklet  93 

to  1984 

112 

10.7 

22 

10.7 

22 

A;     Class  9  Main 

Spiral 

— • 

334 

68.9 

219* 

8.0" 

26'' 

Age  uiass  ij  l>rlages 

Spiral  DooKlets 

51- 

56 

to  1986 

143 

38  ..5 

135 

6.4 

23 

Tape  booklet  90 

Civics 

105 

18 .  J 

55 

18.5 

55 

Tape  booklet  91 

to  1984 

73 

19.2 

36 

19.2 

36 

Tape  booklet  92 

to  1984 

73 

17.5 

36 

17.5 

36 

Tape  booklet  93 

to  iyo4 

/I 

1  "7  *7 
1/  .  . 

56 

17 . 7 

56 

Age  Class  13  Main 

Spiral 

410 

89.5 

221 

6.2*' 

16" 

Age  Class  17  Bridges 

Spiral  booklots 

51- 

56 

to  1986 

107 

43.2 

84 

7.2 

14 

Spiral  booklets 

61- 

67 

to  1984 

119 

59.3 

140 

6.6"= 

10= 

19.8"' 

47"' 

Tape  booklet  90 

Civics 

97 

18.4 

27 

18.4 

27 

Age  Class  17  Main 

Spiral 

312 

104.8 

241* 

7.9*' 

IS'' 

*   Ths  next  hishcst  was  159. 

^   The  per-itea  semple  lizci  £oz  tha  main  lamplci  art  for  th«  major  croaa-acctlonil  laoploa  only.    Thty  do  not  cover  auch  assessment 
components  as  seography  (at  age  class  17)  or  international  mathematics  (at  age  class  13). 
^    for  booklets  61-66 
^    for  booklet  67 
^   The  next  highest  was  200. 
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age-  or  grade- eligible  students  were  considered)  and  also  increased  the  sample 
size  of  grade-oniy-eligible  excluded  students. 


3.6      STUDENT  PARTICIPATION  AND  EXCLUSION  RATES 

Table  3-9  suimnarizes  the  rates  of  exclusion  of  selected  students  and  the 
rates  of  participation  of  invited  students.    The  set  of  invited  students 
consists  of  the  selected  students,  after  removing  the  excluded  students  and, 
in  the  case  of  bridge  samples,  removing  those  students  selected  for  tape 
sessions  who  were  not  age  eligible.     For  a  given  session,  a  makeup  session  was 
called  for  when,  for  various  reasons,  more  than  a  tolerable  number  of  invited 
students  failed  to  attend  the  originally  .scheduled  session  to  which  they  were 
invited.    The  participation  rates  given  in  the  table  express  the  n  inber 
finally  assessed  as  a  percentage  of  those  initiall>  invited  in  the 
participating  schools.    The  rates  for  bridge  and  main  samples  are  combined  for 
age  cl^ss  9  and  age  class  13,  since  the  student  eligibility  criteria  were  the 
same  in  biidge  and  main  samples  for  these  age  classes,     ^or  grade  11/age  17 
samples,  predominantly  eleventh  graders  were  selected,  whereas  for  the  grade 
12/age  17  sample  predominantly  twelfth  graders  wete  selected.     For  each  age 
class.  Table  3-9  presents  comparison  rates  for  the  most  recent  assessment 
(indicated  in  parenthesis)  in  which  a  sizeable  sample  from  the  corresponding 
grade/age  cohort  was  sampled.    Note  that  1988  was  the  first  year  in  which  a 
sample  of  predominantly  twelfth  graders  was  assessed. 


Table  3-9 


Exclusion 

and  Participation  Rates  by  Age 

Class,  Unweighted 

Previojj 

Previous 

Number 

Participation 

Participation 

Excluded  (I) 

Excluded  (Z) 

InvJ  ted 

Rate  (Z) 

Rate  (Z) 

Grade 

4/age  9 

6.3 

3.8  (1984) 

34,535 

92.6 

91.3  (1984) 

Grade 

8Age  13 

5.3 

3.6  (1984) 

54,466 

88.4 

87.3  (1984) 

Grade 

11/age  17 

3.0 

3.4  (1986) 

17,000 

79.2 

78.9  (1986) 

Grade 

12/age  17 

3.7 

41,681 

78.5 

The  major  change  from  previous  assessments  is  in  the  proportion  excluded 
at  lower  ages  (in  1986,  the  exclusion  rate  for  grade  3/age  9  was  3.9  percent, 
while  for  grade  7/age  13  it  was  3.7  percent).    The  reason  for  this  increase  in 
the  rate  of  exclusion,  at  least  for  grade  4/age  19,  is  the  result  of  an 
increase  in  the  proportion  excluded  for  reason  of  limited  English  language 
proficiency.     In  large  part  this  appears  to  be  attributable  to  the  increased 
practice  of  educating  native  Spanish  speakers  in  Spanish  in  elementary 
schools.     In  a  few  selected  schools  for  the  grade  4/age  9  samples,  for 
example,  more  than  50  percent  of  selected  students  were  excluded  from  the 
assessment  for  reason  of  limited  English  language  proficiency.     The  reasons 
for  the  increased  level  of  exclusion  for  13-year-olds  are  less  clear. 


66 


ERiC 


83 


3.7      OVERALL  STUDENT  PARTICIPATION  RATES 


The  combined  impact  of  school  nonparticipation  an1  student  absenteeism 
from  sessions  within  participating  schools  is  summarized  in  ^able  3-10.  Th* 
table  shows  the  percentages  of  students  assessed,  from  among  those  who  would 
have  been  assessed  if  all  initially  selectev^  schools  had  participated,  and  if 
all  invited  students  had  attended  either  an  initial  or  make-up  session.  The 
results  show  that,  consistent  with  earlier  rounds  of  NAEP,  the  overall  level 
of  nonresponse  increases  substantially  with  the  increase  in  age  and  grade  of 
the  students.    Levels  of  nonresponse  at  the  ^welfth  grade,  assessed  for  the 
first  time  in  1988  as  part  of  the  main  sample  for  age  class  17,  were  not 
appreciably  higher  than  for  the  eleventh  grade,  assessed  as  part  of  the  bridge 
sample  for  this  age  class. 

The  procedures  for  substituting  for  nonparticipating  schools,  or 
imputing  for  them  (see  section  8.1.2.1),  and  the  procedures  for  imputing  for 
absent  students  (see  section  8.1.2.4)  were  designed  (so  far  as  feasible)  to 
reduce  the  biases  resulting  from  school  and  student  nonparticipation. 


3.8      SAMPLING  TEACHERS 

The  teacher  questionnaire  was  administered  to  the  reading  teachers  of 
fourth -grace  students  sampled  for  the  main  assessment  of  reading  and  to  the 
writing  teachers  of  eighth-grade  students  sampled  for  the  main  assessment  of 
writing.    The  purpose  of  drawing  these  samples  was  not  to  estimate  the 
attributes  of  the  teacher  population,  but  to  estimate  the  number  (proportion) 
of  students  whose  teachers  had  various  attributes  and  to  correlate  student 
characteristics  and  performance  with  the  characteristics  of  their  teachers 

The  reading  te      ars  of  every  fourth- grade  student  assessed  for  reading 
in  the  main  assessment  (i.e.,  respondents  to  booklets  8-14)  were  identified  in 
each  school.    Up  tc  seven  cf  these  reading  teachers  in  each  school  were 
selected  to  complete  the  teacher  questionnaire;  in  schools  with  more  than 
seven  reading  teachers  with  assessed  students,  a  random  sample  of  five  of 
these  teachers  was  selected.     Every  selected  reading  teacher  was  provided  a 
list  of  all  his  or  her  students  (up  to  a  maximum  of  10)  who  had  been  assesse  I 
for  reading  in  the  main  assessment;  if  more  than  10  students  fitted  the 
criteria,  a  random  sample  of  10  such  students  was  provided.    The  selected 
teachers  were  asked  to  complete  a  questionnaire  about  the  reading  abilities  of 
each  selected  student  and  the  kinds  of  reading  instruction  the  student 
recei^red.     The  analysis  of  the  read-.ng  teanher  questionnaire  data  is  discussed 
in  se  cion  10.3. 

Similarly,  the  writing  teachers  of  every  eighth -grade  student  assessed 
for  writing  in  the  main  assessment  (i.e.,  respondents  to  booklets  1-7)  were 
eligible  to  complete  a  teacher  questionnaire.    Up  to  seven  teachers  were 
sel-^cted  in  each  school  and  each  selected  teacher  was  provided  with  a  list  of 
all  his  or  her  students  (up  to  a  maximum  of  10)  who  had  been  assessed  for 
writing.     The  selected  teachers  were  asked  to  complete  a  questionnaire  about 
the  writing  capabilities  of  each  selected  student  and  about  the  writing 
instruction  the  student  received.     The  analysis  of  the  writing  teacher 
questionn;^ire  data  is  discussed  in  section  11.2. 
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Table  3-10 


Overall  Participation  Rates  (School  and  Student  Combined)  by  Age  Cl 


Age  Class  9 

Main  Samples 


Bridge  Samples 


School  participation  87. 2Z 

Student  participction  92. 2X 

Overall  student  participation  80. 4X 

Number  of  participating  students  8,899 

Overall 

School  participation  88. 3Z 

Student  participation  92. 6Z 

Overall  student  participation  81. 8Z 

Number  of  participating  students  31,991 
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Age  Class  13  Age  Class  17  Overall 


86.  IX 
85. 4Z 
73.  OZ 
92,501 


92. 7Z  78. IX  86. 8X 

90. OX  79. 2X  85.9% 

83.^2  61. 9X  72. 6X 

11.^23  13,460  33,782 


87.81f  31. 4X  86. 3X 

88. 4X  78. 7X  85. 5X 

77. 6X  64. IX  72. 9X 

48,122  46,170  126,283 


School  participation  88. 7X  86. 6X  82  8X 

Student  participation  92  /6X  87. 8X  78  5X 

Overall  student  participation  8^..3X 


Number  of  participating  students  23,092  36,699 


76. OX  65. OX 

32,710 


chapter  4 
ASSESEHENT  INSTRUMENTS^ 


Janet  R.  Johnson 
Educational  Testing  Service 


The  1988  assessment  incorporated  four  tjrpes  of  instruments:  student 
assessment  booklets,  a  questionnaire  about  excluded  students,  a  teacher 
questionnaire  (grades  4  and  8),  and  a  school  characteristics  and  policies 
questionnaire.    The  data  collected  from  these  instruments  are  available  on  the 
1988  NAEP  public-use  data  tapes.    This  chapter  describes  the  assessment 
instrumerns  and  begins  with  a  discussion  of  the  way  in  which  the  item^  were 
organized  into  blocks  to  create  the  student  assessment  instruments. 

4.1      MAIN  SAMPLE  STODENT  ASSESSMENT  INSTRUMENTS 

Student  assessment  booklets  contained  both  cognitive  and  noncognitive 
items.    Cognitive  items  were  used  to  assess  student  achievement  in  ':he  subject 
areas  of  reading,  writing,  civics,  U.S.  history,  and  geography.  Noncognitlve 
items  were  used  to  gather  student  background  and  attitude  information.  Some 
noncognitive  items  were  presented  to  every  studeit;  these  vrere  placed  together 
in  a  block  called  the  common  bac»:ground  block  or  common  core  ^nd  covered  such 
topics  as  race/ethnicity,  levels  of  parental  education,  items  in  the  home, 
homework,  and  television  watching  habits.     Other  noncognit^re  ite^s  were 
specific  to  one  of  the  five  subject  areas.    These  items  appeared  together  in  a 
block  and  were  presented  in  booklets  that  contained  clocks  of  cognitive  items 
rtlatod  to  the  same  subject  area. 

Main  assessment  focused-BIB  booklets  achieved  a  certain  degree  of 
uniformity  in  that  each  booklet-  contained  five  discrete  blucks  of  items:  the 
.irst  hlocK  contained  the  common  core  block  of  background  items;  the  second 
block  contained  the  subject-specific  attitude  items;  the  remaining  three 
blocks  each  contained  cognitive  items  specific  to  a  particular  subject  area. 
Each  stud3nt  at  all  three  grade/ages  was  administered  a  single  booklet. 

Main  assessment  intercorrelation  booklets  resembled  the  focused- ^TB 
booklets  except  that  the  three  blocki.  containing  cognitive  items  each  v.overed 
a  different  subject  area:    reading,  civics,  and  U.S.  history  (as  well  as  some 
geography  at     ade  12/age  17). 


^  The  author  is  heavily  indebted  to  the  National  Assessment  of 
Ei'ucational  Progress  1988  Public-use  Data  Tapes  Version  1.0  User  Guide 
(Rogers,  Kline,  Johnson,  &  Rust,  1989;  and  in  particular  to  Debra  Kline  for 
portions  of  this  chapter. 
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Main  assessment  special  booklets  were  designed  to  allow  special  studies 
relating  to  achievement.    To  measure  the  relationship  between  time  allocated 
to  the  writing  task  and  writing  performance,  each  of  three  booklets  at  each 
grade/age  contained  the  common  background  block  plus  two  writing  blocks,  the 
second  of  which  was  allocated  twice  as  much  time  as  the  first.     In  additf  m, 
there  were  two  special  study  booklets  created  for  grade  8/age  13  and  grade 
12/age  17  that  were  composed  of  the  common  background  block,  the  reading 
noncognitive  block  and  three  cognitive  blocks  that  were  combinations  of 
reading  and  document  literacy  blocks.    These  booklets  were  designed  to  allow 
the  assessment  of  document  literacy  and  its  relation  to  reading  proficiency. 

Block  Timing 

The  common  core  background  question  block  appeared  first  within  every 
booklet.     It  was  read  aloud  to  grade  4/age  9  students  and  took  approximately 
10  minutes  to  complete.    The  o^her  grade/ages  were  given  five  minutes  in  which 
to  romplete  this  section  and  only  the  first  question,  regarding  student 
race/ethnicity,  was  read  aloud. 

At  each  grade/age,  the  common  core  block  w^s  followed  by  a  five -minute 
block  of  subject -specific  background  questions  aboi.»t  students'  experiences  and 
instruction  in  the  subject  area.    All  students  read  these  questions  on  their 
own. 

The  remai.-d'^r  of  each  student  booklet  was  composed  of  three  more  blocks. 
These  were  assembled  from  the  pool  of  cognitive  items  within  a  subject  area. 
Students  at  grade  4/age  9  were  given  blocks  that  vjsre  each  10  minutes  in 
length,  while  the  other  two  grade/ages  were  given  three  15 -minute  blocks. 

The  overall  assessment  time  was  approximately  45  minutes  for  grade  4/age 
9  students  and  "^5  minutes  for  grade  8/age  13  and  for  grade  12/age  17  students. 

Item  Arrangement 

For  the  1988  assessment,  blocks  of  cognitive  items  were  assigned  to 
booklets  in  such  a  way  that  each  subject  area  block  appeared  in  the  same 
number  of  booklets  and  each  possible  paii.  of  blocks  within  a  subject  area 
appeared  in  at  least  one  booklet.    The  uo'^til  pool  of  items  was  so  'arge  that 
no  Individual  student  could  have  received  all  items. 

The  assessment  bocklei  s  rhemso"'ves  were  ordered  in  such  a  way  that 
typically  only  several  scudents  5n  any  assessment  session  in  any  school 
received  the  same  booklet. 

Table  4-1  shows  the  total  number  of  cognitive  blocks  created  for  each 
subject  area  in  the  main  sample. 
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Table  4-1 


Number  of  Main  Sample  Subject  Area  Cognitive  Blocks  Administered 


Subject  Area 


Grade  4 /Ape  9      Grade  8/A   ^  13    Grade  12 /Age  17 


Reading 
Writing 
Civics 
History 


7 
7 
3 
3 


7 
7 
7 
7 


7 
7 
7 
7 
3 
9 
6 
3 


Geography 
Intercorrelation 
Long  Writing 
Document  Literacy 


9 
6 


9 
6 
3 


Total 


20 


31 


34 


Tables  4-2,  4-3,  and  4-4  show  which  subject  area  blocks  were  contained 
in  .**ach  booklet  for  each  grade/age,  how  many  of  each  booklet  were 
administered,  and  the  booklet  response  method  (circling  the  answer  in  the 
booklet,  filling  in  scannable  ovals  in  the  booklet,  or  filling  in  ovals  on  a 
scannable  answer  sheet) . 

Tables  4-5,  4-6,  and  4-7  display  the  composition  of  individual  blocks  of 
items  administered  in  the  main  NAEP  assessment.    These  tables  provide  the  item 
placement  number  within  each  block  as  well  as  the  item  type  (e.g.  background, 
subject,  multiple-choice  cognitive,  or  open-ended  cognitive). 

The  focused- BIB  design  of  these  blocks  and  booklets  generated  a  total  of 
22  different  booklets  for  grade  4/age  9,  36  different  booklets  for  grade  8/age 
13,  and  37  different  booklets  for  grade  12/age  17. 

4.2    BRIDGE  SAMPLE  STUDENT  ASSESSMENT  INSTRUMENTS 

The  1988  assessment  included  four  distinct  bridge  (or  trend)  studies. 
These  are  each  described  below.    Tables  4-8,  4-9,  and  4-10  show  which  subject 
area  blocks  were  contained  in  each  bridge  booklet,  how  many  of  each  booklet 
were  administered,  and  the  booklet  response  method. 

Civics  Bridge  (Booklet  90) 

The  civics  bridge  s^imple  links  the  1988  main  assessment  data  to  data 
from  citizenship/social  studies  assessments  conducted  in  1976  and  1982.  The 
sample  consists  of  one  booklet  each  for  ages  13  and  17.     (Because  there  were 
no  cognitive  civics  items  per  se  from  1976  and  1982  for  age  9,  a  treiid  sample 
for  that  age  was  not  necessary.)    Booklet  90  for  both  ages  contains  the  common 
background  block  (Bl),  the  civics  background  and  attitude  block  from  the  ma^'n 
assessment  (CI),  and  three  blocks  of  cognitive  items  (CS,  CIO,  and  Cli). 
Blocks  Bl  and  CI  each  tr.k  five  minutes  of  assessment  time;  the  three 
cognitive  blocks  were  administered  in  45  minutes.    The  cognitive  blocks 
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Table  4-2 


Booklet  Contents  and  Number  of  Booklets  Administered 
Main  Sample,  Grade  4/Age  9 


Cooomon 

Subject  Area 

Number  o 

Response 

Background 

Background 

Booklet 

ouo.loct  Aroa 

Booklet 

Method! 

Block 

Block 

Cognitive  Blocks 

Admxnxste 

writing 

1 

B 

Bl 

Wl 

W2 

W3 

W5 

896 

2 

B 

Bl 

Wl 

W3 

WA 

W6 

892 

3 

B 

Bl 

Wl 

WA 

W5 

W7 

89A 

k 

B 

Bl 

Wl 

W5 

W6 

W8 

900 

5 

B 

Bl 

Wl 

i:5 

W7 

W2 

901 

6 

B 

Bl 

Wl 

W7 

W8 

W3 

882 

7 

B 

Bl 

Wl 

W8 

W2 

WA 

882 

Roadins 

8 

P 

Bl 

Rl 

R2 

R3 

R5 

889 

9 

Bl 

Rl 

R3 

RA 

R6 

879 

10 

h 

Bl 

Rl 

RA 

R5 

R7 

876 

11 

B 

Bl 

Rl 

R5 

R6 

R8 

885 

12 

B 

Bl 

Rl 

R6 

R7 

R2 

883 

13 

B 

31 

Rl 

R7 

R8 

R3 

88A 

14 

B 

Bl 

Rl 

H8 

R2 

RA 

881 

\S,6,  History 

15 

B 

Bl 

HI 

H2 

H3 

HA 

266A 

Civics 

16 

B 

Bl 

CI 

C2 

C3 

CA 

2652 

Intercorrelation 

17 

B 

Bl 

XI 

H2 

C2 

R2 

869 

(Reading 

18 

B 

Bl 

XI 

R3 

H3 

C3 

88A 

U.S.  History.  Civics) 

19 

B 

Bl 

XI 

CA 

H7 

HA 

885 

Long  Writing 

20 

B 

Bl 

Wl 

W6 

W9 

880 

21 

B 

Bl 

Wl 

W8 

WIO 

877 

22 

B 

Bl 

Wl 

W5 

Wll 

877 

TOTAL  MAIN 

SAMPLE 

BOOKLETS.  . 

23012 

t  B  "  scannablo  bockl'^.t 


ERIC 


30 


Table  4  -3 


Booklet  Contents  and  Number  of  Booklets  Administered 
Main  Sample,  Grade  8/Age  13 


Coomon 

Response 

BackgrouT 

Sub.iect  Aree 

Booklet 

Methodr 

Block 

Writiris 

1 

B 

Bl 

2 

B 

Bl 

3 

B 

Bl 

4 

B 

Bl 

5 

B 

Bl 

6 

B 

Bl 

7 

B 

Bl 

Reading 

8 

A 

Bl 

9 

A 

Bl 

10 

A 

Bl 

11 

A 

Bl 

12 

A 

Bl 

13 

A 

Bl 

14 

A 

Bl 

U.S.  History 

15 

A 

Bl 

16 

A 

Bl 

17 

A 

Bl 

18 

A 

'  I 

19 

A 

1^1 

20 

A 

Bl 

A 

Bl 

Civics 

22 

A 

Bl 

23 

A 

Bl 

A 

Bl 

25 

A 

Bl 

26 

A 

Bl 

27 

A 

Bl 

28 

A 

Bl 

Intercorrelation 

29 

A 

Bl 

(Reading , 

30 

A 

Bl 

U.S.  History,  Civics) 

31 

A 

B7 

Long  Writing 

32 

B 

Bl 

33 

B 

Bl 

34 

B 

Bl 

Document  Literacy 

35 

B 

Bl 

36 

r 

Bl 

Subject  Area 


Block 


Cognitive  Blocks 


Hu  ojer  of 
Booklets 
Admlviistered 


Wl 

W2 

W3 

W5 

866 

Wl 

W3 

W4 

W6 

854 

Wl 

W4 

W5 

W7 

862 

Wl 

W5 

W6 

W8 

360 

Wl 

W6 

W7 

W2 

848 

Wl 

W7 

W8 

W3 

862 

Wl 

W8 

W2 

W4 

859 

Rl 

R2 

R3 

R5 

855 

Rl 

R3 

R4 

R6 

856 

Rl 

R4 

R5 

R7 

849 

Rl 

R5 

R6 

R8 

845 

Rl 

R6 

R7 

R2 

855 

Rl 

R7 

RC 

R3 

795 

Rl 

R8 

R2 

R4 

857 

HI 

H2 

1:3 

H5 

854 

HI 

H3 

H4 

rs 

872 

HI 

H4 

H5 

H7 

851 

HI 

H5 

H6 

H8 

357 

Ul 

H6 

H7 

H2 

865 

HI 

H7 

H8 

H3 

854 

HI 

H8 

H2 

HA 

835 

C 

C2 

C3 

C5 

CI 

C3 

C4 

C6 

C55 

CI 

C4 

C5 

C7 

645 

CI 

05 

C6 

C8 

823 

CI 

C6 

C7 

C2 

^"'5 

CI 

C7 

C8 

C3 

^1 

CI 

C8 

C2 

C4 

863 

XI 

H6 

C7 

R3 

859 

XI 

R5 

H5 

C2 

858 

XI 

C6 

R6 

H2 

873 

Wl 

W6 

wg 

858 

Wl 

W8 

WIO 

869 

Wl 

W4 

Wll 

859 

Rl 

R2 

D2 

R5 

1267 

Rl 

D3 

R7 

D4 

1266 

TOTAL  MAIN 

SAMPLE 

BOOKLETS.  .  . 

31601 

f  A  "  scannable  answer  sheet,  B  ■  scannable  booklet 
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Table  4-4 


Booklet  Contents  and  Number  of  Booklets  Administered 
Main  Sample,  Grade  12/Age  17 


Number  of 

Response 

Background 

Backgroimd 

Booklets 

Subject  Area 

Booklet 

Method t 

Block 

Block 

Cognitive  Blocks 

Administer 

Q 

Bl 

Wl 

W2 

W3 

WS 

827 

2 

Q 

Bl 

Wl 

W3 

WA 

W6 

ez5 

•J 

Q 

Bl 

HI 

rf  J 

W7 

813 

Q 

Bl 

HI 

WS 

W6 

W8 

827 

5 

3 

Bl 

HI 

W6 

W7 

U2 

821 

6 

B 

Bl 

Wl 

W7 

W8 

W3 

31S 

7 

B 

Bl 

Wl 

W8 

W2 

WA 

802 

Koading 

o 
o 

A 

Bl 

Rl 

R2 

R3 

RS 

822 

g 

Bl 

Rl 

R3 

RA 

R6 

81A 

A 

Bl 

Rl 

n't 

R7 

82A 

Bl 

Rl 

RS 

R6 

R8 

826 

1  9 

Bl 

Rl 

R6 

R7 

R2 

823 

13 

A 

Bl 

Rl 

R7 

R8 

R3 

838 

14 

A 

31 

Rl 

R8 

R2 

RA 

821 

U.S.  History 

IS 

k 

Bl 

HI 

H2 

H3 

HS 

831 

16 

A 

Bl 

HI 

B3 

HA 

H6 

827 

17 

A 

Bl 

HI 

HA 

HS 

H7 

830 

18 

A 

Bl 

HI 

BS 

H6 

H8 

820 

19 

A 

Bl 

HI 

no 

Q  ' 

H2 

20 

A 

Bl 

HI 

H7 

ad 

B3 

82A 

21 

Bl 

HI 

H8 

H2 

HA 

819 

Civics 

22 

A 

Bl 

CI 

C2 

C2 

CS 

818 

23 

A 

Bl 

CI 

03 

CA 

C6 

809 

2A 

A 

Bl 

CI 

CA 

C5 

C7 

817 

25 

A 

Bl 

CI 

C5 

C6 

C8 

807 

26 

A 

CI 

C6 

C7 

C2 

810 

27 

A 

Bl 

CI 

C7 

C8 

ZZ 

81A 

28 

A 

Bl 

CI 

C8 

C2 

CA 

808 

Geography 

29 

A 

Bl 

Gl 

G2 

G3 

GA 

2AA6 

Intf/rcori  elation 

30 

A 

Bl 

XI 

R2 

H6 

GA 

817 

(Reading,  U.S.  Hifator^' 

31 

A 

Bl 

XI 

H2 

R5 

CA 

81S 

Civics,  Geography) 

32 

A 

Bl 

.1 

G2 

C6 

R8 

806 

Long  Writing 

33 

B 

Bl 

HI 

W6 

W9 

799 

34 

B 

Bl 

Wl 

W8 

WIO 

811 

35 

B 

Bl 

HI 

W4 

Wll 

820 

Doctiment  Literacy 

36 

B 

Bl 

Rl 

R2 

D2 

RS 

1217 

37 

B 

Bl 

Rl 

D3 

R7 

DA 

1208 

TOTAL  MAIN  SAMPLE  BOOKLETS.  .   .  32710 


f  A     scannable  answer  sheet,  B  >■  scannable  booklet 


74 

92 


Table  4-5 

1988  Main  Sample  Block  Information,  Grade  4/Age  9 


Total 


urQer  or         uruer  ol 

Tnt-a  1 
1 0  UdX 

r\n  on  -  OTlHoH 
UU  c  il    C  ilUtsU 

DacKgrouno      uogni cive 

locai 

oogiix  c.  X  ve 

K^U^llL  U  X V c 

Block 

Type 

items  items 

Items 

I  tems 

Items 

1-21 

21 

1-10 

1 0 

X  V 

P  A  o  /I  ^  v>  n 
£\.6  aClJLilg 

1  -1  A 

1  4 
x*+ 

14 

t\.e  aomg 

1-7 

7 

7 

QP/i 

P  O  O  ^  1  CT 

1-7 

7 

7 

KeaQing 

1  -1  S 

X  J 

x.^ 

1 
X 

QPA 

P  A  o  ^  f  rr 

KcaQing 

IL                     1  -1 

1  L 

XH 

13 

QP7 

KeaQing 

■;  - 1  s 

X    X  J 

QPA 

KeaQing 

1-11 

X  -  X  X 

X  X 

X  X 

1 

X 

9W1 

Wrt.  Bkgd. 

1-10 

10 

Writing 

1 

1 

X 

X 

9W3 

Writing 

1 

i 

1 

1 

9W4 

Writing 

1 

i 

1 

1 

9w5 

Writing 

1 

i 

1 

1 

QUA 

-/WO 

Ur  i  f-  i  n  CT 

Wi.  J.  C>J.il^ 

1 

1 

1 

QU7 

Wl.  J.  U  J-ilg 

1 

X 

1 

1 

Uritinp 

W  1.     i»  J.il^ 

1 

1 

1 

1 

QtJQ 

LiOng  wriuing 

1 
X 

X 

1 

X 

9W10 

Lone  Writine 

1 

1 

1 

1 

9W11 

Long  Writing 

1 

1 

1 

1 

9C1 

Civics  Bkgd. 

1-9 

9 

9C2 

Civics 

1-15 

15 

15 

9C3 

Civics 

1-16 

16 

16 

■•C4 

Civics 

1-20 

20 

20 

Hist.  Bkgd. 

1-9 

9 

9H2 

History 

1-15 

15 

15 

9H3 

History 

1-15 

15 

15 

9H4 

History 

1-15 

15 

15 

9X1 

Intercorr.  Bkgd 

.  1-10 

10 

m 


lERJC 
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Table  4-6 


.988  Main  Sample  Block  Informatiou,  Gr  de  8/Age  13 

Total 


wXUcX     \f  L.                 V/XUCX  UX 

Totol 

Open-ended 

BArkp^round  Cognitive 

Total 

Cognit  ive 

Cognitive 

DlOCK 

lype 

XL  ems  Items 

T  t" omc 
X  tciiio 

T  t"  OITIC 

T  t  pni<» 

13B1 

floromoTi  Bk.P'd 

1-21 

21 

RflcT  Bkpd 

1-20 

20 

14 

R  ois f1 4  n  CT 

1-14 

14 

13R3 

RpAfii  xw 

1-9 

9 

9 

Rp Afl^  n CT 

1-10 

10 

10 

Rp i  ncT 

1-14 

14 

14 

1  '^Rfi 

RpAfii 

1-18 

18 

18 

R  o  A  H    n  C7 

iKxCCiUXil^ 

1-19 

19 

19 

1  ^I\0 

1-15 

15 

15 

1 

13wl 

wrt.  Bkgd. 

1  OA 

9  A 

1  QT70 

T.7  V>  4  f*  4  i-t  rr 

wri uing 

1 
X 

X 

1 JWJ 

wri uing 

J. 

X 

1  QTTA 
i  JW^f 

wri uing 

1-9 
J.  *  z 

0 

wri uing 

X 

Uv>4  t*4  n  rr 

wn  uing 

TJv>  4  t*  4  1-k  rr 

wriuing 

X 

wx  J.I.  j.ng 

1 

X 

1 

1  1T7Q 

T  ^<               T.7      4  ^4  ir\  n 

JLfOng  writing 

1 

X 

X 

13W10 

Lons  Urltlnc 

1 

1 

1 

I 

13W11 

Lonp  Writinp 

1 

1 

X  £.mJ 

25 

r4v4  r< 

O  X  V  Xi^o 

1-26 

26 

26 

UX V XCd 

i-24 

24 

24 

p4 iy4  i-» c 

u  X  y  X  c  d 

1  •  30 

30 

30 

P4  vf  r»c 

O  X  V  Xi^O 

1-27 

27 

27 

C* 4  ^v4  ^  ^ 

ui  vies 

X  - 

24 

24 

O  X  V  Xi«  O 

1-22 

22 

22 

O  X  V  XV«  O 

1 

1 

1 

1 

nlSC.  DKga. 

1  97 
1  -  Z  / 

97 

13H2 

History 

1-26 

26 

26 

13H3 

History 

1-26 

26 

26 

13H4 

History 

1-26 

26 

26 

13H5 

History 

1-26 

26 

26 

13H6 

History 

1-28 

28 

28 

13H7 

History 

1-28 

28 

28 

13H8 

History 

1 

1 

1 

1 

13D^ 

Doc.  Lin. 

1-10 

10 

10 

7 

13D3 

Doc.  Lit. 

1-14 

14 

14 

10 

13D4 

Dec.  Lit. 

1-13 

13 

13 

10 

13X1 

Intercorr.  Bkgd.  1-25 

25 

ERIC 
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Table  4-7 


1982  Main  Sample  Block  Information,  Grade  12/Age  17 

Total 


Order  of        Order  of 

Total 

Open-endei 

Background  Cognitive 

Total 

Cognitive 

Cognitive 

Block 

Type 

terns  Items 

Items 

Items 

Items 

17B1 

Common  Bk^d. 

1-33 

33 

17R1 

Rdg.  Bkgd. 

1-32 

32 

17R2 

Reading 

1-14 

14 

14 

17R3 

Reading 

1-11 

11 

11 

1 

17R4 

Reading 

1-19 

19 

19 

17R5 

Reading 

1-14 

14 

14 

17R6 

Reading 

1-15 

15 

15 

1 

17R7 

Reading 

1-15 

19 

19 

17R8 

Reading 

1-18 

18 

18 

17W1 

Wrt.  Bkgd. 

1-35 

35 

17W2 

Writing 

1 

1 

1 

1 

17W3 

Writing 

1 

1 

1 

1 

17W4 

Writing 

1-2 

2 

2 

2 

17W5 

Writing 

1 

1 

1 

1 

17W6 

Writing 

1 

1 

1 

1 

17W7 

Writing 

1 

1 

1 

1 

17W8 

Writing 

1 

1 

1 

1 

17W9 

Long  Writing 

1 

1 

1 

1 

17W10 

Long  Writing 

I 

1 

1 

1 

17W11 

LonR  Writing 

1 

1 

1 

17C1 

Civics  Bkgd. 

1-34 

34 

17C2 

Civics 

1-26 

26 

26 

17C3 

Civics 

1-26 

26 

26 

17C4 

Civics 

1-25 

25 

25 

17C5 

Civics 

1-27 

27 

27 

17C6 

Civics 

1-24 

24 

24 

17C7 

Civics 

l-?.2 

22 

22 

17C8 

Civics 

1 

1 

1 

1 

17H1 

Hist.  Bkgd. 

1-36 

36 

17H2 

History 

1-28 

28 

28 

17H3 

History 

1-25 

25 

25 

17H4 

Histo;  y 

1-25 

25 

25 

17H5 

History 

1-26 

26 

26 

17H6 

History 

1-28 

28 

28 

17II7 

History 

1-28 

23 

28 

17H8 

History 

1 

1 

1 

1 

17G1 

Geog .  Background 

1-28 

28 

17G2 

Geography 

1-26 

26 

26 

17G3 

Geography 

1-26 

26 

26 

17G4 

Geography 

i-26 

26 

26 

17D2 

Doc.  Lit. 

1-10 

10 

10 

7 

17D3 

Doc.  Lit. 

1-14 

14 

14 

10 

17D4 

Doc.  Lit. 

1-13 

13 

13 

10 

17X1 

Intercorr.  Bkgd. 

1-36 

36 

77 


Table  4-8 


Booklet  Contents  and  ^  jnber  of  Booklets  Administered 
Bridge  Samples,  Age  Class  9 


BRIDGE  TO  1984,  GIL       4/AGE  9 


Cooxoon 

Subject  Ar^d 

Nun)bcr  of 

Response 

Backsround 

Background 

Booklets 

^ethodf 

Block 

Hlock 

CORnitivo  BlOi,*xs 

Admlnistored 

Roadin^  and 

C 

CC 

C         I  Q 

68A 

Hritins 

52 

C 

CC 

I 

HER 

879 

9J 

L\> 

UNO 

5k 

c 

CC 

G        0  E 

853 

55 

c 

CC 

M         'f  K 

861 

56 

c 

CC 

V  R 

851 

TOTAL  1984 

BRIDGE  BOOKIETS.  . 

5188 

BRIDGE  TO  1986, 

AGE  9 

CooxDon 

Subject  Area 

Hunger  of 

Response 

Backsround 

Backsround 

Booklets 

Subi«ct  Ar«a 

Book  jt 

Methodf 

Block 

Block 

CoKnltlvo  Blocks 

Admin  Ifi  tor  ed 

Rc^dins,  Hathoa«tic«« 

91 

B 

Bl 

1 

Rl       Ml  Si 

1X74 

and  Sci«nc« 

92 

B 

Bl 

S2      R2  r:3 

1240 

93 

B 

Bl 

1 

M2       S3  R3 

1197 

TOTAL  1966 

BRIDGE  BOOKLF'3. 

3711 

t  B      scsnnable  booklet,  C     circIed'ansHer  booklet 

{  Subject  area  backsround  questions  are  included  in  the  cogritive  blocjcs  'or  this  booklet. 
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ERIC 


Table  4-9 


Booklet  Contents  and  Number  of  Booklets  Administered 
Bridge  Samples,  Age  Class  13 


BRIDGE  TO  1984,  GRADE  8/AGE  13 


Sub.iect  Area 

Readins  and 
Writing 


Respond 


5  )klet 

H*>triod| 

Block 

51 

c 

cc 

52 

c 

cc 

53 

c 

cc 

54 

c 

cc 

55 

c 

cc 

56 

c 

cc 

Couxaon      Subject  Area 
Background  Background 
Block 


Cognitive  Blocks 


h 
C 

r, 

N 
G 
G 


K 
L 
£ 
C 

o 
J 


D 
Q 
R 
D 
£ 
P 


Kumber  o£ 
Booklets 
Administered 

907 
915 
924 
927 
906 
921 


TOTAL  1984  BRIDG£  BOOKLETS. 


5500 


CIVICS  BRIDGE,  AGE  13 


Sub.iect  Area 
Civics 


Respon: 
Booklet     Method f 

90  B 


Cocaaon      Subject  Area 
r  ickground  Background 
Block  Block 


Bl 


Cl 


Crgnitive  Blocks 
C9       CIO  Cll 


TOTAL  CIVICS  BRIDG£  B00KL£TS. 


Kuoiber  of 
Booklets 
Administered 

1938 
1933 


BRIDGE  TO  1986,  AGE  13 


Subject  Area 

Reading,  Mathematics, 
and  Science 


Cocxnon      Subject  Area 
Response      Background  Background 
Booklet     Method f  Block  Block 


91 
93 


Bl 
Bl 
Bl 


CoRnitive  Blocks 


Rl 

s:, 

M2 


Ml 
R?- 
S3 


S. 
M3 
R3 


Number  o£ 
Booklets 
Administered 

1405 
1281 
1256 


TOfAL  iS86  BRIDGE  BOOKLETS. 


3942 


f  B  ■  scannable  booklet,  C  ■  circled-answer  booklet 

I  Subject  area  background  questions  are  included  in  the  cognitive  blocks  for  this  booklet 


79 


ERLC 


n 
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Table  4-10 

Booklet  Contents  and  Number  of  Booklets  Administered 
Bridge  Samples,  Age  Class  17 


BRIDGE  TO  1984,  GRADE  11/AGE  17 


Subject  Ar«a 
Readins  aod  Hritins 


Response 
Booklet  Mathodf 


51 
52 
53 
54 
55 
56 


C 
C 
C 

c 
c 
c 


Coczxson      Subject  Area 
Background  Background 
Block  Block 


CC 

cc 

CC 

cc 
cc 
cc 


CoKnitive  Blocks 


M 
C 
H 
N 
G 
G 


K 
L 
E 
C 
O 
J 


D 
Q 
R 
D 
E 
P 


Number  of 
Booklets 
Administered 

774 
768 
770 
777 
/T* 
762 


TOTAL  1984  BRIDGE  BOOKLETS. 


4622 


BRIDGE  TO  1986,  GRADE  11/AGE  17 


Sub.ltfct  Area 

Reeding,  KathecDatics, 
and  Science 


U.S.  aistory 


Comson 

Subject  Area 

Nuznber  o£ 

Response 

Background 

Background 

Booklets 

Booklet 

Methodf 

Block 

Block              Cognitive  Blocks 

Administered 

61 

B 

Bl 

1  R2 

M4 

H2 

799 

62 

B 

Bl 

1  Ml 

R2 

M9 

786 

63 

B 

Bl 

1  SI 

Sll 

Rl 

798 

64 

B 

Bl 

1  S2 

S4 

R5 

779 

65 

B 

Bl 

1  S3 

R6 

H3 

780 

66 

B 

Bl 

1  R3 

R4 

R2 

761 

67 

B 

Bl 

H2 

H3 

H4 

2349 

TOTAL  1986  BRIDGE  BOOKLETS.  . 

7052 

CIVICS  BRIDGii,  AGE  17 


Subject  Area 
Civics 


Response 
Booklet     Method f 

90  B 


CocxDon      Subject  Area 
Background  Background 
Block  Block 


Bl 


CI 


CoRnitivo  Blocks 
C9       CIO  Cll 


Number  o£ 
Booklets 
Administered 

1786 


TOTAL  CIVICS  BRIDGE  BOOKLETS. 


1786 


t  B  •*  scennable  booklet,  f  »  circled-answer  booklet 

{  Subject  area  background  questions  are  included  in  the  cognitive  blocks  for  this  booklet. 
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contained  Items  that  were  used  in  the  1982  assessment  of  citizenship  and 
social  studies  as  well  as  several  items  that  vere  newly  developed  for  the  1988 
assessment  for  the  purpose  of  maintaining  consistent  timing  across  blocks  of 
Items . 

In  order  to  match  the  1976  and  1982  assessment  characteristics,  age -only 
samples  of  students  were  defined  using  the  1976  and  1982  age  definitions,  13- 
year-olds  were  tested  in  the  fall,  17 -year- olds  were  tested  in  the  spring,  and 
administration  of  the  civics  booklets  was  paced  with  an  audiotape. 


Bridge  to  1984  (Reading  and  Writing,  Booklets  51-56) 

The  1984  bridge  samples  are  comparable  to  the  1984  main  assessment  of 
reading  and  writing.     Samples  of  students  at  grade  4/age  9,  grade  8/age  13, 
and  grade  11/age  17  (the  same  age  and  grade  target  populations  assessed  in 
1984)  were  determined  using  the  1984  age  definitions  and  were  assessed  during 
the  same  time  of  the  year  as  in  1984.     The  assessment  booklets  (six  at  each 
grade /age)  were  adniinister«d  without  an  audiotape. 

At  each  grade/age,  booklets  51  through  56  all  contain  a  common 
background  block  (CC)  and  three  cognitive  blocks,  either  two  reading  and  o'- 
writing,  or  one  reading  and  two  writing.    All  blocks  are  identical  to  tho» 
used  in  the  1984  assessment.    Blocks  C,  D,  E,  and  G  are  writing  blocks;  .^ck^ 
H,  J,  K,  L,  M,  N,  0,  P,  Q,  and  R  are  reading  blocks.     At  grade  4/age  9,  block 
V,  a  double-length  block,  contains  both  writing  and  reading  items.  All 
writing  and  reading  blocks  also  contain  subject -related  background  and 
attitude  questions. 

The  common  background  block  was  administered  in  approximately  15  minutes 
to  grade  4/age  9  and  six  minutes  to  ihe  other  grade/ages;  14  minutes  were 
allowed  to  complete  each  cognitive  block  (except  for  block  V,  which  was 
allowed  28  minutes) . 


Bridge  to  1986  fcr  Ages  9  and  13  (Reading,  Mathematics,  and  Science, 
Booklets  91-93) 

These  bridge  samples  are  comparable  to  those  used  for  the  measurement  of 
trends  In  reading,  mathematics,  and  science  in  1986.     Three  assessment 
booklets  each  were  administered  to  samples  of  9 -year -old  and  13 -year* old 
students.    The  age-only  samples  were  obtained  using  the  same  age  definitions 
and  times  of  testing  as  were  used  for  both  the  1984  main  assessment  and  the 
1986  bridge  to  1984.    The  mathematics  and  science  blocks  were  administered 
using  an  audiotape;  the  reading  blocks  were  administered  without  an  audiotape. 

The  contents  of  booklets  91  through  93  are  identical  to  booklets  used  in 
the  1986  assessment.    Each  booklet  contains  a  common  background  block  (Bl^  and 
three  blocks  of  cognitive  items — one  reading  block  (R1-R3),  one  science  block 
(S1-S3),  and  one  mathematics  block  (M1-M3).    The  cognitive  blocks  also  contain 
subject-related  background  and  attitude  questions. 
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The  common  background  block  took  approximately  15  minutes  at  age  9  and 
six  minutes  at  age  13.  The  reading  blocks  took  13  minutes  each  at  age  9  and 
16  minutes  at  ages  13  and  17. 

Bridg3  to  1986  for  Grade  11/Age  17  (Reading,  Mathematics,  Science,  and  U.S. 
History — Booklets  61-67) 

These  bridge  samples  are  comparable  to  the  1986  main  assessment.  Seven 
assessment  booklets  were  administered  to  samples  of  grade  11/age  17  students. 
The  grade/age  samples  were  obtained  using  the  same  age  definitions  (not 
calendar  year)  and  time  of  testing  (spring)  used  for  both  the  1984  and  1986 
nain  assessments.    The  booklets  were  administered  without  an  audiotape. 

Booklets  61  through  66  all  contain  a  common  background  block  (Bl)  and 
three  blocks  of  cognitive  items  in  various  combinations  of  reading  (R1-R6) , 
science  (S1-S4,  Sll) ,  and/or  mathematics  (M1-M3,  M9) .    Booklet  67  contains  a 
common  background  block  (Bl)  and  three  history  blocks  (H2'H4) .    The  cognitive 
blocks  also  contain  subject- related  background  and  attitude  ques**ions.  The 
booklets  were  constructed  and  cycled  for  administration  using  the  BIB  spiral 
design. 

The  common  background  block  took  five  minutes  of  assessment  tLne;  each 
cognitive  block  took  16  minutes  of  assessment  time. 


4.3  QUESTIONNAIRES 

In  addition  to  the  student  assessment  booklets,  three  questionnaires 
were  administered  to  collect  data  about  school  characteristics,  teachers 
associated  with  sampled  students,  and  students  excluded  from  the  assessment. 


The  Teacher  Questionnaires 

NAEP  gathered  information  on  curricula  and  teaching  msthcds  from  two 
distinct  samples  of  teachers.     The  teacher  questionnaires  were  administered  to 
a  sample  of  the  reading  teachers  of  fourth -grade  students  who  were  assessed  in 
reading  for  the  main  assessment  and  a  sample  of  the  writing  teachers  of 
eighth- grade  students  who  were  assessed  in  writing  for  the  main  assessment. 
(The  method  by  which  teachers  were  sampled  is  described  in  Chapter  3.)  These 
teachers  completed  a  questionnaire  that  surveyed  years  of  teaching  experience, 
course  ^^urriculfi.  use  of  classroom  time,  instructional  practices,  home  jrk 
assignments,  and  teaching  materials  used. 

Note:    The  purpose  of  this  questionnaire  was  to  collect  additional 
information  about  students  by  gathering  information  about  their  teachers,  not 
to  describe  the  attributes  of  the  teacher  population. 
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The  School  Characteristics  and  Policies  Quest ionxiaire 


The  school  characteristics  and  policies  questionnaire  was  completed  by 
the  school  principal  or  his  or  her  representative  for  every  school  included  in 
any  of  the  1988  samples.    The  questionnaire  was  used  to  gather  information 
about  school  administration,  staffing  patterns,  special  programs,  subject 
requirements,  a  "d  school  resources. 


The  Excluded  Student  Questionnaire 

This  questionnaire  was  completed  by  school  personnel  for  every  student 
selected  for  inclusion  in  the  NAEP  sample  who  was  unable  to  respond  to  items 
because  he  or  she  was  classified  by  the  school  as  being  limited  in  English 
language  proficiency,  mildly  mentally  retarded  (educable) ,  or  functionally 
disabled.     The  questionnaire  was  used  to  gather  information  about  special 
education,  language,  and  other  student  programs. 
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Chapter  5 
FIELD  ADMINISTRATION 


Nancy  Caldwell  and  Renea  Slobasky 
Westat,  Inc. 


As  a  subcontractor  to  ETS,  Westat,  Inc.,  was  responsible  for  field 
activities  that  included  contacting  selected  districts  and  schools, 
administering  the  assessment  sessions,  and  delivering  completed  booklets  and 
questionnaires  to  ETS.    This  chapter  summaries  the  Westat  field  organization 
and  operations  for  the  1988  assessment.     Details  of  field  administration 
activities  are  available  in  the  Westat  Report  on  Field  Operations  and  Dati 
Collection  Activities — NAEP  1988  (Caldwell,  Moore,  &  Slobasky,  1989). 

This  chapter  begins  with  an  overview  of  the  field  organization,  followed 
by  discussions  of  the  training  of  supervisors;  procedures  for  gaining 
cooperation  of  districts  and  schools;  the  supervisors'  responsibilities, 
including  making  arrangements  for  the  assessment,  sampling,  conducting  the 
assessments,  and  collecting  questionnaires;  and  the  results  of  the 
assessments. 


5.1      ORGANIZATION  OF  THE  FIELD  OPERATION 

The  field  operation  was  organized  around  a  core  group  of  home  office  and 
field  staff  supplemented  by  additional  staff  in  preparation  for  and  during  the 
winter  and  spring  assessment.    The  core  home  office  staff  consisted  cf  the 
field  director  and  assistant  field  director.    Throughout  the  study,  the  field 
director  coordinated  all  activities  in  the  home  office  related  to  field 
operations.    The  assistant  field  director  coordinated  the  distribution  of 
materials  and  the  receipt  of  reporting  forms.    During  the  fall,  the 
supervisor.s  were  assigned  for  routine  reporting  to  the  assistant  field 
director.    When  the  size  of  the  field  staff  tripled  for  the  winter/spring 
assessment,  two  more  assistant  field  directors  were  added  to  the  home  office 
staff  to  share  telephone- reporting  and  trouble -shooting  responsibilities. 

The  sample  of  schools  for  the  1988  assessment  was  selected  from  eligible 
schools  in  selected  geographic  areas.    These  aLeas  were  a  county  or  group  of 
counties,  called  primary  sampling  units  (PSUs) .    The  sample  of  areas  consisted 
of  94  PSUs,  including  a  core  group  of  64  PSUs  used  for  the  fall  bridge  samples 
plus  30  additional  PSUs  for  the  winter  and  spring  samples.     The  field  staff 
was  similarly  organized  around  a  core  group  of  11  supervisors  who  were 
responsible  for  fall  pre- assessment  and  assessment  activities  in  the  64  core 
PSUs.    Prc-assessment  activities  in  the  94  PSUs  for  the  winter  and  spring 
assessments  were  conducted  by  37  supervisors,  including  the  original  11,  in 
the  late  fall  after  the  awarding  of  the  new  contract. 
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For  the  winter  and  spring,  when  the  number  of  PSUs  increased  and  the 
worklcad  in  terms  of  both  the  number  of  schools  and  students  to  be  assessed 
within  each  PSU  also  increased  significantly,  supervisory  regions  were 
rearranged  and  expanded  in  number  from  11  to  37.    During  the  fall,  each 
supervisor  was  responsible  for  assessments  in  an  average  of  5-6  PSUs.  During 
'Jhe  winter  and  spring,  supervisors  worked  in  an  average  of  1-2  PSUs. 

Of  the  11  fall  supervisors,  six  had  worked  on  NAEP  during  at  least  one 
of  the  past  assessments.    With  the  awarding  of  the  new  contract  and  expansion 
of  work  for  the  winter  and  spring,  26  additional  supervisors  were  required. 
To  fill  these  positions,  we  rehired  five  supervisors  who  had  worked  on  NAEP  in 
the  past.    To  find  the  remaining  supervisors  for  the  winter/spring,  we  first 
searched  Westat  supervisory  files,  and  then  recruited  from  outside  the 
company. 

During  the  field  period,  supervisors  were  replaced  in  four  of  the  37 
supervisory  regions.    Two  of  the  supervisors  resigned  because  of  personal 
problems,  the  other  two  because  the  position  was  more  demanding  than  they  had 
anticipated. 

Each  district  supervisor  was  responsible  for  a  variety  of  different 
tasks.    During  the  fall  pre- assessment  phase,  the  11  supervisors  contacted 
school  districts  with  schools  scheduled  to  be  assessed  in  che  fall  to  follow 
up  on  introductory  materials  that  had  been  mailed  earlier  explaining  the 
assessment  program.     During  these  calls,  supervisors  also  scheduled 
introductory  meetings  with  representatives  of  the  sampled  schools.  The 
purposes  of  the  introductory  meetings  were  to  explain  the  program  in  greater 
detail  and  to  set  a  schedule  for  the  assessment  in  each  school.    While  in  the 
area  conducting  meetings,  the  supervisors  recruited  exercise  administrators  to 
work  with  th*%in  in  administering  the  assessment  sessions. 

Except  for  the  introductory  materials  mailed  hy  ETS,  no  contact  was  made 
with  districts  containing  schools  scheduled  only  for  the  winter  and/or  spring 
until  the  new  contract  was  awarded  in  September,  1987.     In  October,  26 
supervisors  and  two  alternates  were  hired  and  trained.    They  followed  the  same 
process  described  above  in  contacting  the  winter/spring  school  districts  and 
conducting  introductory  meetings  during  the  months  of  November  and  December, 
1987.    Also  during  this  time,  the  11  fall  supervisors  who  were  in  the  process 
of  conducting  the  fall  assessments  expanded  their  schedules  to  include  the 
contacts  with  the  districts  in  their  regions  in  which  there  were  winter/spring 
schools. 


5.2      SUPERVISOR  TRAINING 

The  11  fall  supervisors  came  to  Bethesda,  Maryland,  for  a  four-day 
training  session  from  September  2-5,  1987.    Also  in  attendance  were 
representatives  from  the  ETS  Princeton  and  regional  offices  who  were  there  to 
help  the  supervisors  gain  cooperation  froi.1  schools  and  districts.  The 
training  was  conductea  by  the  Westat  project  director  and  field  director.  ETS 
Princeton  office  staff  also  made  presentations  and  provided  explanatory  notes 
throughout  the  session. 
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With  the  awarding  of  the  new  NAEP  contract,  26  additional  supervisors 
and  two  alternates  were  hired  and  brought  to  Rockville,  Maryland,  for  training 
on  October  26-27,  1987.    This  training  session,  conducted  by  the  Westat 
project  director  and  field  director,  focused  only  on  scheduling  and  conducting 
introductory  meetings  for  the  winter/spring  assessments. 

To  assist  supervisors  in  conducting  their  introductory  meetings,  a  slide 
presentation  with  a  script  for  the  supervisor  to  read  was  developed  and 
implemented  for  the  first  time  during  the  fall  introductory  meetings  for  the 
1988  assessment.    The  slides  and  script  formalized  the  supervisors' 
presentations  on  background  information  of  National  Assessment,  the  general 
stages  of  sampling  schools  and  students,  the  role  of  the  school  in  the 
assessment,  criteria  for  student  eligibility,  and  questionnaires  to  be 
completed  by  school  personnel.     Supervisors  reported  that  the  slide 
presentations  were  well  received  at  introductory  meetings  attended  by  several 
school  representatives.    However,  they  often  chose  not  to  use  the  presentation 
during  small,  one -on- one  meetings. 

Procedures  for  conducting  the  winter/spring  assessments  were  the  focus 
of  a  three-day  training  session  attended  by  all  supervisors  on  December  16*18, 


5.3      OBTAINING  COOPERATION  OF  SCHOOL  DISTRICTS  AND  SAMPLE  SCHOOLS 
5.3.1  Preliminary  Contacts 

During  June,   ^nly,  and  August  1987,  while  Westat  was  recruiting 
supervisors  and  developing  materials,  ETS  was  making  prelin.inary  contacts 
preparatory  to  obtaining  school  cooperation.    The  schedule  of  these 
preliminary  activities  is  outlined  below  and  discussed  in  mor-  detail 
following  the  outline. 


1987. 


Date 


Contact 
Made  By 


Activity 


June  29 


ETS 


An  initial  letter  was  mailed  to  Chief  State  School 
Officers  informing  them  that  schools  within  their 
states  had  been  selected  for  NAEP. 


July  7 


ETS 


A  secord  letter  was  sent  to  Chief  State  School 
Officers  containing  a  list  of  th'^  school  districts  and 
private  schools  selected  within  the  state. 


July  22 


ETS 


An  initial  letter  and  a  NAEP  report  were  mailed  to 
superintendents  of  public  and  parochial  schools  and 
principals  of  private  schools  selected  for  the  entire 
1988  assessment  within  the  state. 
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August  5       ETS  A  followup  letter  and  NAEP  materials  were  mailed  to 

superintendents  and  private  school  principals.  The 
materials  included 

•  a  cover  letter  explaining  that  the  supervisor 
would  contact  them  to  discuss  NAEP  and  schedule 
an  introductory  meeting; 

•  a  list  of  the  selected  sc^.ools  in  the  official's 
jurisdiction; 

•  a  fact  sheet  on  NAEP;  and 

•  examples  of  assessment  items. 


As  can  be  seen  from  the  outline,  recruiting  of  schools  for  NAEP  actually 
began  in  June,  once  the  sample  of  schools  had  been  selected  and  their 
corresponding  school  districts  identified.    ETS  contacted  the  Chief  State 
School  Officers  in  each  state  and  asked  them  to  notify  the  school  district 
superintendents.     In  July,  ETS  sent  a  letter  to  the  superintendents  and  heads 
of  private  schools  inviting  their  participation.    Informational  materials  on 
NAEP  and  a  list  of  the  sampled  schools  in  the  district  were  also  sent.  These 
Initial  contacts,  which  were  completed  prior  to  supervisor  training,  paved  the 
way  for  the  telephone  contacts  to  follow. 

Once  che  supervisors  and  ETS  regional  office  staff  had  been  trained, 
they  began  working  :o  obtain  cooperation.    The  schedule  of  these  contacts 
follows. 


Contact 
Date  Made  by 

Sept.  8-  District  sapervisors 
Oct.  9  for  fall  assessment, 

with  ETS  assistance 


Activity 

Calls  were  made  to  districts  with  schools 
selected  for  fall  assessments.    The  calls 
were  made  to  introduce  the  1988 
assessment,  establish  participation,  and 
make  arrangements  for  introductory 
meetings.    The  supervisor  filled  out  the 
Introductory  Meeting  Form(s)  and  the 
School  Update  Form  and  summarized  the 
conversation  in  the  Results  of  Contact 
fonn.    Copies  of  the  forms  were  sent  to 
the  home  office.    Districts  that  had 
schools  selected  for  winter/spring  as  well 
as  fall  assessments  were  told  that  further 
contact  concerning  winter/spring 
assessments  would  occur  later. 
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Sept.  8-       ETS  Calls  were  made  to  districts  with  four  or 

Oct.  9  more  schools  selected  only  for 

winter/spring  assessments,  and  were  told 
that  the  supervisor  would  contact  the 
district  in  October-November. 


Sept.  8-       Westat  home  office  A  confirmation  letter  was  sent  to 

Oct.  9  superintendents  after  the  Introductory 

Meeting  Form  was  received  from  the 
supervisor.    Principals  of  selected 
schools  within  the  superintendent's 
district  were  sent  a  package  containing 


•  a  cove-  memo  introducing  the  study 
and  confirming  the  meeting  date; 

•  a  memo  to  the  principal  giving  the 
assessment  schedule  and  outlining 
the  school's  role; 

•  the  principal's  questionnaire:  and 

o         other  informational  materials  (the 
same  as  those  sent  to 
superintendents) . 


Sept.  14- 
Oct.  9 

Oct.  28- 
Dec.  4 


Oct.  28- 
Dec.  15 


Fall  district 
supervisors  and  ETS 

District  supervisors 
for  winter/spring 
assessment,  with  ETS 
assistance 


Westat  home  office 


Introductory  meeting<;  were  conducted  for 
schools  in  the  fall  assessment. 

Calls  were  made  to  districts  with  schools 
selected  for  winter/spring  assessments. 
The  supervisor  filled  out  the  Introductory 
Meeting  form,  the  School  Update  Form,  and 
th^  Results  of  Contact  form.     If  the 
district  had  schools  in  the  fall 
assessment,  the  primary  purpose  of  the 
call  was  to  make  arrangements  for 
introductory  meeting  with  winter/spring 
schools;  the  School  Vpdate  Form  would  have 
alrfc/<dy  boen  filled  out  by  the  fall 
supervisor. 

A  confirmation  letter  was  sent  to 
superintendents  when  the  Introductory 
Meeting  Form  was  received  from  the 
supervisor.     Principals  of  selected 
schools  in  the  superintendent's  district 
were  sent  a  package  containing  information 
similar  to  that  sent  to  principals  with 
schools  in  the  fall  assessment. 
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Contacting  districts  to  solicit  cooperation  and  schedule  the  meetings 
was  primarily  the  responsibility  of  the  district  supervisor,  unless  we  had 
some  reason  to  believe  that  getting  the  cooperation  of  a  particular  district 
or  school  was  going  to  be  a  problem.     In  those  cases,  ET3  regional  or  national 
staff  made  the  initial  contacts.    An  ETS  representative  also  made  the  initial 
call  to  all  districts  coi;taining  four  or  more  schools  in  the  assessment. 

During  these  calls  to  establish  cooperation  and  to  set  up  the 
Introductory  meeting,  supeirvisors  also  updated  our  information  on  schools  in 
the  district.    Topics  raised  with  school  district  staff  included  school 
creation  and  closing,  and  changes  in  enrollment  and  grade  span.     Three  forms 
were  used  to  record  this  informati  n:     the  School  Update  Form,  the 
Introductory  Meeting  Form,  and  the  Results  of  Contact  form.     The  originals  of 
these  forms  were  mailed  to  the  home  office  and  then  used  as  the  basis  for 
mailing  packages  of  materials  to  the  persons  scheduled  to  attend  the  meeting. 
Information  from  the  School  Update  Form  was  used  to  revise  and  update 
information  in  the  home  office  files  on  the  schools  in  the  sample.  New 
schools  identified  were, also  given  a  chance  to  be  selected  for  the  study. 

There  were  basically  two  waves  of  telephoning  and  introductory 
meetings — one  for  districts  with  schools  in  the  fall  assessment  and  another 
for  districts  with  schools  only  in  the  winter/spring.     For  the  districts  that 
had  schools  in  the  winter/spring  as  well  as  the  fall  assessment,  a  second 
telephone  call  was  made  to  arrange  for  an  introductory  meeting  for 
winter/spring  schools,  and  to  obtain  any  updated  infoimation  about  ;»chools  in 
the  district  not  obtained  by  the  fall  supervisor. 


5.3.2  Schools  Added  to  the  Original  Sample 

Due  to  a  variety  of  factors,  described  in  The  1988  National  Assessment 
of  Educational  Progress — Sampling  and  Weighting  Proced'^ares ,  Final  Report 
(Rust,  Bethel,  Burke,  &  Hansen,  1990),  it  was  sometimes  necessary  to  add 
schools  to  the  original  sample.     Since  the  process  of  adding  schools  did  not 
begin  until  October,  while  introductory  meetings  were  taking  place,  the 
procedures  for  contacting  and  gaining  cooperation  from  these  schools 
necessarily  differed  from  those  described  for  the  original  sample.     For  the 
added  schools,  ETS  first  mailed  a  letter  to  the  district  superintendents  and 
heads  of  private  schools.    Then,  the  district  supervisor  telephoned  the 
contact  person  in  the  superintendent's  office  and  asked  him  or  her  to  notify 
the  sample  schools.     If  convenient,  th^  supervisor  then  met  with  school 
representatives  in  person.     If  it  was  not  possible  to  meet  in  person  because 
of  scheduling  problems,  the  supervisor  conducted  the  introductory  meeting  by 
telephone.     ETS  regional  and  national  staff  pr>vided  assistance  as  needed  in 
contacting  districts  and  schools. 
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5.3.3  Results  of  School  Cooperation  Effort^ 

There  were  1,776  schools  originally  selected  for  one  or  more  grade/aj^e 
samples  in  1988.    This  is  comparable  to  the  1,682  schools  selected  for  1984, 
but  less  than  the  original  sampl    for  1986  of  2,309.    The  1986  sample  was  made 
larger  to  accommodate  inclusion  of  the  language  minority  component  and  because 
of  the  number  of  different  subject  areas  assessed  that  year. 

Table  5-1  presents  the  results  of  the  school  cooperation  effort.    Of  the 
1,776  schools  originally  selected  for  the  1988  assessment,  1,412  cooperated. 
An  additional  88  would  have  cooperated,  but  they  did  not  have  any  eligible 
students  enrolled  and  230  refused.     The  other  46  schools  were  closed  or  out 
of -scope. 

The  cooperation  rate  (86.7  percent)  for  the  1988  assessment  was  about 
the  same  as  in  1986  but  less  than  in  1984  (88.1  percent). 

As  Table  5-2  indicates,  the  school  cooperation  rata  varied  among  the 
three  age  classes  and,  more  noticeably    ^etween  the  main  NAEP  and  bridge 
assessments.     For  the  main  NAEP  assessments,  the  rate  varied  from  88.7  percent 
for  grade  4/age  9  schools  to  82.8  percent  for  grade  12/age  17  schools. 

The  bridge  assessments  for  age  class  13  were  held  in  the  fall  cf  1987 
and  had  the  highest  school  cooperation  rate  (92.7  percent).    The  bridge 
assessii^ents  for  age  class  9  were  held  in  the  winter  and  had  a  school 
cooperation  rate  of  87.2  percent.    The  bridge  assessments  for  age  class  17, 
held  in  the  spring  of  1988,  had  a  school  cooperation  rate  of  only  78.1 
percent. 


5.4      OVERVIEW  OF  THE  SUPERVISORS'  MAJOR  TASKS 

During  the  assessment  phase  of  the  project,  the  supervisors  wera 
responsible  for  carrying  out  the  ta'sks  listed  below. 

•  Recruit  and  train  exercise  administrators. 

•  Check  each  shipment  from  ETS  for  quantities  of  assessment  booklets 
and  excluded  student  questionnaires  and,  in  the  winter/spring, 
teacher  questionnaires.     Check  shipment  from  Westat  for  the  other 
supplies  needed. 

•  Review  the  Session  Assignment  Forms  for  the  PSU  and  formulate  a 
plan  for  scheduling  assessment  activities  in  each  school. 


^The  summary  numbers  presented  in  Table  5-1  prefer  to  individual  schools, 
whereas  Tables  5-2,  16-4,  and  16-5  refer  to  grade/age  samples.     Since  a  school 
may  be  selected  for  more  than  one  grade/age  sample,  che  school  appears  in  the 
counts  for  each  of  the  appropriate  assessment  samples  in  Tables  5-^,  16-4,  an J 
16-5. 
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Table  5-1 


Summajry  of  NAEP  1988  School  Participation 

Number  of  Schools 


Total  original  sample 

1,776 

Out-of-range  or  closed 

46 

No  eligibles  enrolled 

88 

District  refused 

149 

School  refused 

81 

Cooperating 

1,412 

Cooperation  rate* 

OD  .  /  A 

[1986] 

86. 4X 

[1984] 

88.  IX 

Replacement  sample  for  refusals 

86 

Out-of-range  or  closed 

4 

No  eligibles  enrolled 

0 

District  refused 

24 

School  refused 

11 

Cooperating 

47 

Table  5-2 

NAEP  1988  School  Cooperation  Rate*  by  Age  Class 

and  Tjrp 

Number 

Number 

of 

Cooperating 

Refusals 

Cooperation 
Rate 


Age  class  9  main  NAEP 
Winter  bridge 

Age  class  13  main  NAEP 
Fall  bridge 

Age  class  17  main  NAEP 
Spring  bridge 


3^7 
\34 

399 
173 

304 
114 


42 
23 

68 
16 

66 
34 


88. 7X 
87. 2X 

86. 6X 
92. 7X 

82. 8X 
78.  IX 


*  Cooperation  rate 


 Cooperating  +  no  eligibles  enrolled  

Cooperating  +  no  eligibles  enrolled  +  refused 
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Contact  each  school  one  week  before  the  assessment  week  and 
confirm  the  exact  days  for  the  assessment;  remind  the  school 
coordinator  that  the  Student  Listing  Forms  must  be  completed 
before  the  supervisor  arrives. 

Contact  each  school  two  days  before  the  sample  selection  was  to 
begin  to  be  sure  the  school  was  prepared. 

Contact  exercise  administrators  to  inform  them  when  and  where  to 
meet  on  the  first  day  in  a  PSU. 

Collect  school  characteristics  and  policies  questionnaire  from 
each  school. 

Complete  the  sample  selection  of  students  in  each  school  a  few 
days  to  a  week  before  the  assessment  began.    Arrange  with  the 
school  coordinator  the  exact  time  and  place  for  each  session  and 
complete  an  Administration  Schedule  for  each  session. 

Prepare  an  excluded  student  question.iaire  for  each  student  who  had 
been  sampled  from  the  Student  Listing  Form  bur  had  been  excluded 
from  the  assessment  by  the  school.    Give  the  school  coordinator 
the  excluded  student  questionnaires. 

Supervise  and  coordinate  the  assessment  process. 

In  the  winter/spring,  complete  sample  selection  of  teachers  for 
the  teacher  questionnaires  and  distribute  the  teacher 
questionnaires . 

Perform  quality  control  checks  on  each  exercise  administrator. 

Collect,  or  have  the  exercise  administrator  collect,  completed 
excluded  student  questionnaires  (and  teacher  questionnaires  in  the 
winter/spring) . 

Complete  Roster  of  Questionnaires  for  school  characteristics  and 
policies  questionnaire  and  excluded  student  questionnaire  and,  as 
appropriate,  the  Teacher  Survey  Roster  for  the  teacher 
questionnaire . 

On  a  daily  basis,  take  back  completed  work  from  exercise 
administrator  and  distribute  new  materials  for  the  next  day. 

Make  arrange jients  for  makeup  sessions  as  needed. 

Review  each  booklet  to  make  sure  that  all  coding  was  accurate. 
Compare  the  front  cover  to  the  Administration  Schedule  and  make 
any  corrections  needed. 

Complete  the  School  Worksheet. 
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Ship  materials  to  ETS  and  mail  appropriate  report£  to  Westat. 


•  Call  the  field  director  immediately  if  there  was  any  problem 
completing  the  assessments  in  a  PSU. 

5.5  HAKING  ARRANGEMENTS  FOR  THE  ASSESS^tENTS 

To  make  arrangements  for  the  assessments,  the  supervisors  worked  with 
the  schools  to  accomplish  the  following: 

•  Schedule  the  date  and  time  for  the  assessment  sessions. 

•  Make  sure  that  appropriate  space  would  be  available. 

•  Identify  a  scliool  staff  member  to  be  the  school  coordinator  who 
would  work  with  the  supervisor  on  assessment  arrangements. 

•  Arrange  for  the  necessary  number  of  exercise  administrators  and, 
if  necessary,  additional  school  staff  to  monitor  the  sessions. 

•  If  the  school  required  any  parental  notification  or  permission, 
make  sure  that  the  appropriate  letters  or  forms  were  distributed 
to  the  students. 

•  Select  the  sample  and  prepare  all  assessment  materials. 

5.6  SELECTING  THE  SAMPLE  OF  STUDENTS 

At  the  introductory  meetings,  schools  received  instructions  and  forms  to 
bp  used  to  prepare  lists  of  eligible  students.    The  instructions  specified  who 
should  be  listed  and  the  information  (birth  date,  gender,  race/cithnicity ,  and 
grade)  to  be  provided.    The.  forms  were  to  be  used  if  the  school  v-^hose  to  do 
so.    Since  experience  in  previous  years  showed  that  an  increasing  number  of 
schools  could,  and  preferred  to,  produce  computer -generated  lists  of  eligible 
students,  a  set  of  instructions  for  computer  listing  of  students  was  also  made 
available.    The  Instructions  made  it  clear  that  computer  lists  were  perfectly 
acceptable  as  lon^  as  the  necessary  information  on  each  student  was  included. 

Two  weeks  prior  to  a  school's  assessment  date,  the  supervisor  contacted 
the  school  coordinator  to  make  sure  that  the  lists  of  eligible  students  were 
prepared  and  that  all  arrangements  were  set  as  agreed.    The  supervisor  then 
visited  the  school  (or  district  office)  to  select  the  sample  of  students  to  be 
as.^ecsed.    The  time  interval  between  the  selection  of  the  sample  and  the 
assessment  varied,  depending  on  several  factors;  however,  the  average  elapsed 
time  was  about  a  week.     Schools  with  large  assessments  ana  those  requiring 
parental  notification  generally  preferred  that  the  sampling  be  done  as  early 
as  possible  to  give  them  time  to  make  final  arrangements. 
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In  scheduling  sample  selection,  the  supervisors  had  to  balance  several, 
often  competing,  concerns.    Their  own  time  constraints  and  travel  schedule 
limited  theit  flexibility  in  scheduling  sampling.    Also,  the  schools' 
preference  for  early  sampling  had  to  be  balanced  against  the  importance  of 
having  an  up-to-date  sample.    The  more  tims  that  elapsed  between  sampling  and 
assessment,  the  more  likely  that  the  sample  wou'^d  include  students  had 
dropped  out  and  that  it  wo.ild  exclude  ne\»ly  enrolled  students.    These  problenis 
were  most  severe  in  the  large  urban  high  schools. 

Once  the  supervisor  hari  re*'5.ewed  the  lists  for  completeness  and 
accuracy,  he  or  she  selected  the  sample  following  the  instructions  on  the 
Session  Assignment  Form.    In  iiiost  cases  the  supervisors  worked  from  lists 
prepared  by  the  school  and  did  the  seimpling  in  the  schools.     In  some 
instances,  districts  prepared  computer  lists  for  their  schools  and  some  of 
these  districts  required  that  the  sampling  be  done  in  the  district  office 
rather  than  at  the  school. 

Because  of  the  complexity  of  the  sampling,  tiupervisors  were  required  to 
do  all  sampling  themselves,  although  they  could  use  exercise  administrators  to 
help  check  the  sampling  lists  and  to  fill  out  forms.    Having  exercise 
administrators  paiTticipate  in  the  sampling  was  very  helpful  to  the  supervisors 
and  expedited  the  sampling  process. 

Once  sampling  had  been  completed,  the  supervisors  and  exercise 
administrators  could  make  out  the  lists  of  student^:  to  attend  each  asses  ;ment 
session.     The  supervisor  reviewed  the  plans  for  the  assessment  with  the  school 
coordinator  before  making  out  the  lists  or  Administration  Schedules.    At  this 
time,  and  for  the  first  time,  the  exact  number  of  students  sampled  was  known. 
Working  with  the  coordinator,  the  supervisor  updated  the  plans  for  the 
assessment  and  determined  which  students  would  go  to  which  sessions. 
Sometimes  the  coordinator  had  very  specific  ideas  about  the  organization  of 
the  assessment.     In  elementary  schools,  for  example,  the  usual  preference  was 
for  all  students  in  a  particular  teacher's  class  to  be  assessed  together  and, 
if  possible.  In  their  own  classroom. 

The  supervisor  had  a  great  deal  of  flexibility  in  arranging  spiral 
assessments  in  main  NAEP  schools  to  suit  the  needs  of  the  school.    There  was 
not  as  much  flexibility  with  tae  bridge  spiral  or  tape  sessions  because  each 
student  was  sampled  for      particular  type  of  session  had  to  attend  the  session 
when  that  booklet  type  v*js  administered. 

Once  the  <^rrangements  had  been  set  and  the  Administration  Schedules 
filled  out,  the  supervisor  distributed  the  excluded  student  questionnaires. 

5.7      DISTRIBUTING  AND  COLLECTING  NAEP  QUESTIONNAIRES 

The  school  characteristics  and  policies  quest.lonnaire,  excluded  student 
questionnaire,  and  teacher  survey  questionnaire  were  distributed  in  the 
schools  to  be  completed  by  school  personnel. 
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All  schools  were  mailed  the  school  characteristics  and  policies 
questionnaire  by  Westat  prior  to  the  assessment.    This  form  was  to  be  filled 
out  by  the  principal  or  another  staff  member  knowledgeable  about  the  school's 
administrative  policies  and  staff  characteristics.    The  supervisors  picked  up 
the  questionnaire  when  at  the  school  for  sampling  or  for  the  assessment. 

An  excluded  student  questionnaire  was  to  be  filled  out  for  every  rtudent 
who  was  sampled  for  the  assessment  but  excluded  by  the  school.     Schools  could 
exclude  students  with  limited  English  speaking  ability  and  those  who  were 
mildly  mentally  retarded  (edacabl^^^  or  functionally  disabled,  if  in  the 
judgment  of  school  staff  or  it  a  review  of  the  school  records  indicated  that 
they  were  unable  to  take  ths  assessment. 

Main  NAEP  schools  sslected  for  the  grade  4/age  9  and  grade  8/age  13 
assessments  were  asked  to  participate  in  a  post- assessment  teacher  survey. 
The  fupervisor  used  the  completed  Administration  Schedules  to  identify  those 
fourth-grade  students         completed  reading  booklets  and  eighth-grade  students 
who  completed  writing  booklets.    To  identify  the  teachers  for  the  survey,  the 
school  coordinator  was  asked  to  Identify  the  reading/English  teachers  of  those 
students.    These  were  the  teachers  selected  for  the  survey.    The  supervisor 
gave  the  school  coordinator  a  teacher  questionnaire  to  distribute  to  each 
selected  teacher.    The  supervisor  (or  exercise  administrators)  returned  to 
these  schools  to  pick  up  the  teacher  questionnaires  a  few  days  after  the 
assessment. 

The  supervisor  attempted  to  collect  all  the  completed  questionnaires  on 
the  assessment  day  except  tor  the  teacher  questionnaires.     If  the 
questionnaires  were  not  ready  on  the  assessment  day,  and  it  was  convenient  for 
the  supervisor  or  an  exercise  administrator  to  return  to  the  school  later  to 
pick  up  the  questionnaires,  they  would  do  so.    Otherwise,  the  supervisor  gave 
the  coordinator  a  postage -paid  envelope  to  be  used  tc  mail  the  forms  to  ETS. 

5.8  PREPARING  REPORTS  AND  SHIPPING  MATERIALS 

Once  the  assessments  wore  finished  in  a  school,  the  supervisor  and 
exercise  administrators  edited  the  booklets,  filled  out  the  necessary  forms 
and  shipped  the  booklets  and  forms  to  ETS.    A  copy  of  all  forms  was  sent  to 
Westat  so  that  progress  in  the  field  could  be  monitored. 

5.9  RESULTS  OF  THE  FALL  ASSESSMENT 

Table  5-3  shows  data  on  the  n amber  of  students  who  were  sampled,  invited 
and  assessed  during  the  fall  assessment  of  13-year-olds. 

The  original  sample  included  13,494  students.    Of  these,  806  students 
were  sampled  but  excluded  from  participation  by  the  school  because  of  they  had 
limited  English  speaking  ability,  were  mildly  mentally  retarded  (educable)  or 
were  functionally  disabled.    The  rate  of  exclusion  (6.0  percent)  is  somewhat 
higher  than  in  the  previous  assessment,  when  it  was  5.4  percent.    The  number 
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Table  5-3 


Students  Sampled,  Invited,  and  Assessed 
During  the  Fall  Assessment 


Sample  Type 


Number  of 

Bridge 

Civics 

Bridge 

to  1986 

Booklets 

Students 

to  1984 

Bridge 

91 

92 

93 

Invited 

6,189 

2,165 

1,534 

1,377 

1,423 

Assessed 

5,500 

1,938 

1,405 

1,281 

1,256 

Percent  Assessed 

88. 8X 

89. 5Z 

91. 6Z 

93. OZ 

88. 3Z 

Overall 

12,688 
11.380 

89. 7Z 


Table  5-4 

Students  Sampled,  Invited,  and  Assessed 
During  the  Winter  and  Spring  A.ssessments 

  Winter  and  Spring 


Age 

Age 

Age 

1988 

Class  9 

Class  13 

Class  17 

Overall 

Number  sampled 

36,961 

44,118 

60,833 

141,912 

Number  excluded 

2,253 

2,213 

2,065 

6,531 

Number  invited 

34,708 

41,905 

58,768 

135,381 

Number  assessed 

31,911 

36,699* 

46,170 

114,780* 

Percent  astjessed 

91. 9Z 

87. 6Z 

78. 6X 

84. 9X 

*  Number  Includes  5,098  13-year-old  students  who  were  assessed  as  part 
of  the  International  Assessment  of  Mathematics  and  Science  (see  A  World  of 
Differences:    An  International  Assessment  of  latbematics  and  Science. 
Technical  Report    [King,  Bertrand,  &  Dupuis,  1989)). 
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of  students  invited  to  the  assessment  was  12,G88.  Of  these,  11,380  actually 
were  assessed. 

The  overall  attendance  rate  (89.7  percent)  is  the  same  as  in  1986  (89.7 
percent)  and  slightly  higher  than  in  1984  (87.3  percent^  and  1982  (85.5 
percent) . 

5.10    RESULTS  OF  THE  WINTER/SPRING  ASSESSMENT 

Table  5-4  provides  information  on  the  number  of  students  sampled, 
invited  to  assessment,  and  assessed  during  the  winter  and  spring. 

Of  the  almost  142,000  students  sampled  for  assessment,  6,531,  or  4.6 
percent  were  excluded  by  their  schools.    Of  those  students  invited  to 
assessment,  just  under  85  percent  were  assessed.    During  the  20  weeks  of  the 
winter  and  spring  assessment,  114,780  students  were  assessed.    Overall,  the 
assessment  rates  for  the  three  age  classes  have  remained  relatively  stable 
over  the  past  few  assessments. 
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chapter  6 

OVERVIEW  OF  1988  NAEP  MATERIALS  PROCESSING  ANP  DATABASE  CREATION 


John  L.  Barone 
Educational  Testing  Service 


Chapters  6.1  through  6.7  detail  the  receipt,  processing,  and  final 
disposition  of  the  1988  NAEP  assessment  materials  at  ETS.    These  processes 
resulted  in  the  integration  of  all  respondent  data  collected  during  the  1988 
assessment  into  a  NAEP  database — a  database  that  ensures  data  quality  and 
provides  for  efficient  analysis  and  reporting. 

Chapters  6.1  through  6.4  describe  the  methods  used  to  transcribe  the 
materials  to  computer -readable  form.    Chapter  6.5  discusses  the  quality 
control  processes  that  were  followed  during  data  transcription  and  provides  a 
siammary  of  quality  control  error  analyses.    Chapter  6.6  discusses  the 
integration  of  the  transcribed  NAEP  data  into  the  NAEP  database/information 
system  that  was  used  for  data  analysis  and  reporting.     Chapter  6.7  describes 
the  database  products  that  result  from  the  NAEF  information  system. 

This  chapter  describes  the  flow  and  evolution  of  the  operational 
procedures  used  to  process  the  1988  NAEP  data,  and  provides  some  detail  on  the 
amounts  of  materials  that  were  processed. 

Also  described  in  this  chapter  are  the  three  distinct  data  units  that 
make  up  the  intsgrated  NAEP  database: 

1)  the  item  information  database,  which  contains  information  about 
every  assessment  item  used  in  a  1988  assessment  booklet; 

2)  the  restricted-use  data  files,  which  contain  all  data  collected 
from  the  1988  NAEP  respondents;  and 

3)  the  public-use  data  files,  which  contain  a  nonconfidential  subset 
of  the  restricted-use  data  files,  and  are  avail/ible  to  external 
users  via  the  1988  NAEP  public-use  data  tapes  package. 

The  flow  of  materials,  creation  of  data  files,  and  creation  of  the  N\EP 
database  are  depicted  as  an  ordered  set  oi:  processes  that  are  applied  either 
to  the  assessment  materials  or  to  the  transcribed  data.    Chapters  6.1  through 
6  7  describe  these  processes  in  detail. 
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6.  OA  THE  198d  ASSESSMENT:  SCOPE  OF  WORK 


The  scope  of  the  effort  required  to  process  the  1988  assessment 
materials  is  evidenced  by  the  following  numbers. 

For  the  1988  assessment, 

•  more  than  130,000  assessment  booklets  or  questionnaires  were 
received  and  processed. 

This  processing  included 

•  optically  scanning  more  than  1  million  double-sided 
pages  from  test  booklets  and  questionnaires; 

•  professionally  scoring  more  than  300,000  student  responses  from 
130  open-ended  items; 

o         manually  key-entering  and  verifying  more  than  15,000  assessment 
booklets; 

using  the  NAE?  minicomputer-based  transcription  system  to  track, 
audit,  edit,  and  resolve  more  than  22  million  characters  of 
inforaatlon; 

•  selecting  and  comparing  a  quality  control  sample  of  more  than 
160,000  characters  of  transcribed  data  to  the  actual  responses  in 
assessment  booklets; 

•  cataloging  more  than  1.5  mill  ton  characters  of  information  on  a 
total  of  3,800  assessment  items  and  derived  variables,  as  part  of 
a  comprehensive  item  information  database; 

•  developing  a  public -use  data  tape  package  containing  more  than 
160  million  characters  of  useful  information. 


These  numbers  alone  indicate  the  staggering  size  of  1988  NAEP  materials 
processing  and  database  operations.    However,  the  full  extent  of  this  effort 
becomes  clearer  when  one  considers  that  over  90  percent  of  the  data 
transcription  activities  described  in  this  chapter  were  completed  within  six 
months,  with  a  conservatively  estimated  accuracy  rate  of  fewer  than  2.5  errors 
for  every  10,000  characters  of  information  transcribed. 


6*0.2  NAEP  DATA  PROCESSING  SYSTEMS 

Materials  processing  and  database  creation  for  the  1988  assessment 
closely  parallelea  the  processes  used  In  the  1984  and  1986  assessments.  This 
allowed  the  use  of  in-place,  proven  operational  procedures  and  computer 
svstems.    This  fact  will  be  emphasized  and  highlighted  throughout  the 
Mowing  chapters. 
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A  major  improvement  in  the  1988  assessment  was  the  introduction  of 
scannable  answer  sheets.    This  innovation  greatly  reduced  the  amount  of  paper 
handling  and  scanning  required  to  process  assessment  materials.    The  NAEP  data 
systems  vere  adapted  to  accept  this  type  of  answer  doctunent. 

The  large  volume  of  collected  data  and  the  complexity  of  the  NAEP 
design,  with  its  spiraled  distribution  of  many  booklets,  required  the 
development  and  use  of  NAEP-specif ic  data  entry  and  management  systems, 
including  carefully  planned  and  well-defined  editing,  quality  control,  and 
auditing  procedures.    This  chapter  discusses  the  original  1984  design  and 
Implemei.tation  of  these  systems,  and  the  adaptation  and  use  of  these  sy?tems 
and  processes  as  applied  to  the  1986  and  1988  assessments.    The  result  was 
effective,  res^^a&ive  data  management  procedures  that  ensured  the  quality  and 
integrity  of  NAEP  data.    And,  a  NAEP  database  that  met  the  orip'.nal  objectives 
of  integrity  and  usefulness,  while  exceeding  stringent  standards  for  accuracy 
and  quality. 

Types  of  Assessment  Booklets  and  Answer  Documents  Used  in  NAEP 

NAEP  data  processing  flow  and  systems  are  determined  to  a  large  extent 
by  the  type  of  as'-.assment  booklets  and  answer  documents  that  need  to  be 
processed.    As  a  consequence  of  the  1988  NAEP  design,  three  types  of  answer 
documents  were  used  and  processed  for  the  assessment,     key- entered  booklets, 
scannable  booklets,  and  scannable  answer  sheets.    The  evolution  of  these 
response  documents  is  described  below. 

In  the  1984  assessment,  students  circled  their  responses  in  their  test 
booklets.    These  responses  were  then  manually  transcribed,  item  by  item,  to  a 
computer  file.     In  1986,  because  of  a  higher  volume  of  data  and  a  shorter  time 
period  for  processing,  a  new  type  of  '  ooklet  was  introduced  in  which  students 
filled  in  ovals  to  indicate  their  responses.     Each  page  of  these  booklets  was 
entered  into  a  computer  scanning  device  to  :,reate  the  data  file.    The  NAEP 
data  transcription  systems  were  modified  to  accept  the  output  of  the  scatiOing 
devices. 

As  part  of  the  continual  effort  to  improve  the  efficiency  of  NAEP  data 
processing,  the  use  of  separate,  scannable  answer  sheets  was  introduced  in  the 
1988  assessment.    Main  sample  students  in  grade  12/age  17  and  grade  8/age  13 
filled  in  ovals  corresponding  to  their  responses  to  multiple-choice  items  on  a 
separate  answer  sheet,  instead  of  in  the  test  booklet.     Because  the  answer 
sheets  were  only  one  or  two  pages  in  length — ^versus  a  test  booklet,  the  ""ength 
of  which  could  be  up  to  30  double-sided  pages — the  amount  of  paper  handling 
and  scanning  required  to  process  these  samples  was  substsrt/ally  reduced.  The 
NAEP  data  transcription  systems  were  adapted  to  accommodate  the  new  answer 
sheets  in  addition  to  the  scannable  booklet  and  direct  entry  methods  that  were 
already  in  place. 

For  those  booklets  tha^  contained  open-ended  response  items,  students 
recorded  tho.ir  written  responses  on  the  answer  document.     Later,  professional 
scorers  at  ETS  scored  the  items  and,  depending  on  the  type  of  answer  document, 
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wrote,  circled,  or  filled  in  an  oval  for  the  students'  scores  on  the  answer 
document.    The  document  was  then  transcribed  to  a  computer  file. 

Students  in  some  of  the  1988  samples  did  not  receive  separate  answer 
sheets.    Main  sample  grade  4/age  9  students  filled  in  ovals  next  to  responses 
in  the  test  booklet,  because  there  was  some  concern  th^t  separate  answer 
sheets  would  be  confusing  to  them.     Bridge  sample  students  (students  in 
samples  selected  to  link  1988  results  with  past  results)  were  given 
instruments  that  matched  those  used  in  the  corresponding  previous  assessments. 

In  addition  to  the  student  assessment  booklets,  three  questionnaires 
were  administered  to  collect  data  about  school  characteristics,  teachers 
associated  with  sampled  students,  and  students  excluded  from  the  assessment. 
The  excluded  student  questionnaire  was  a  scannable  document.    The  school  and 
teacher  questionnaires  were  manually  transcribed,  item  by  item,  to  a  computer 
file. 


6,0.3  PROCESS  FLOW  OF  NAEP  MATERIALS  AND  DATABASE  CREATION 

Figure  6-1  is  a  flow  diagram  that  shows  the  conceptual  framework  of 
ordered  processes  that  were  applied  to  the  NAEP  materials  and  data  files.  The 
vertl3S*al  line  through  the  center  of  the  figure  divides  the  processes  into  two 
sets — ^processing  assessment  materials  and  database  creation — both  of  which  a  e 
described  below. 

The  processes  represented  by  rectangular  boxes  in  the  flow  diap^'am  were 
performed  at  ETS  on  the  paper  materials  or  computer  files.     The  three 
processes  enclosed  in  rounded  boxes  (assembling  the  sample  of  schools, 
planning  and  conducting  the  field  administration,  and  deriving  the  sampling 
weights)  were  performed  by  Westat  and  are  discussed  respectively  in  Chapters 
3,  5,  and  8.    Two  Westat  reports,  the  Report  on  Field  Operations  and  Data 
Collection  Activities — NAEP  1988  (Caldwell,  Moore,  6c  F'obasky,  1989)  and  The 
1988  National  Assessment  of  Educational  Progress — Sampling  and  Weighting 
Procedures,  Final  Report  (Rust,  Bethel,  Burke,  6c  Hansen,  1990)  discuss  the 
field  operations  and  sampling  procedures  in  detail. 
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Processing  Assessment  Materials 

The  left  side  of  Figure  6-1  depicts  the  flow  of  NAEP  printed  materials. 
Chapter  6,1  describes  this  flow  in  detail  .and  discusses  how  information  from 
the  field  rosters,  schedules,  and  worksheets  was  used  to  control  the 
processing  of  materials.     The  figure  follows  the  path  of  each  assessment 
instrument  (student  test  booklets,  school  characteristics  and  policy 
questionnaires,  teacher  questionnaires,  and  excluded  student  questionnaires), 
absentee  rosters,  school  worksheets,  and  administration  schedules  as  they  are 
tracked  through  the  appropriate  processes  that  result  in  the  final  integrated 
NAEP  database. 

The  following  is  a  brief  description  of  the  materials  processing 
activities  as  shown  on  the  left  side  of  Figure  6-1.     Each  description  refers 
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Figure  6-1 
Data  Flow  Overview 
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the  reader  to  the  section(s)  or  chapter(s)  J  \  which  the  activity  is  discussed 
in  detail. 

Field  Administration  is  the  cOi.^act  and  monitoring  of  the  NAEP 
assessment  in  the  schools.    Chapter  5  summarizes  this  process. 

Receipt  of  Materials  refers  to  receipt  and  processing  of  assessment 
materials  at  ETS.     Section  6.1.1  describes  the  procedures  and  forms  that  were 
used  to  check  and  verify  the  receipt  of  documents  from  the  field.    It  also 
discusses  the  follow-up  procedures  that  were  initiated  when  discrepancies  were 
identified  and  the  subsequent  assembling  of  NAEP  materials  for  further 
processing  and  data  transcription. 

The  Professional  Scoring  process  was  carried  out  for  responses  to  ooen- 
ended  items  for  reading,  writing,  civics,  U.S.  history,  mathematics,  and 
science.    Chapter  6.2  describes  the  items,  types  of  scoring  used,  scoring 
operation,  reliability  checks,  and  resolution  of  scoring  discrepancies.  Entry 
and  editing  of  these  data  ars  discussed  in  sections  6.1.4  and  6.4.4. 

Data  Transcription  Systems  refers  to  the  methods  used  to  transcribe  NAEP 
materials  into  computer -readable  form.    The  transcription  method  used  for  each 
type  of  NAEP  instrument,  is  discussed  in  Chapter.  6.1.     Chapter  6.3  describes 
the  design,  structure,  and  development  of  the  data  entry  system  used  to 
transcribe  most  of  the  NAEP  materials  to  computer  files;  it  also  discusses  the 
tracking  and  audit  mechanisms  that  were  built  into  the  system  t.  ensure  that 
all  data  were  properly  processed  and  accounted  for. 

Originally  implemented  for  the  1984  assessment,  NAEP's  data 
transcription  system  has  proven  to  be  accurate,  efficient,  and  flexible.  In 
1984,  mar  aal  key  entry  and  verification  was  the  primary  method  of  entering 
data.    In  the  1986  assessment,  the  system  was  modified  to  accept  scannable 
booklets  as  the  main  source  of  input.     For  the  1988  assessment,  the  system  was 
modified  to  accept  scannable  answer  sheets  as  a  third  method  of  data  entry. 

Editing  refers  to  the  ETS  procedures  that  ensur  .d  the  corr^^ctness  and 
integrity  of  the  NAEP  data  files  by  (1)  validating  every  field  of  NAEP  data 
that  was  entered  into  computer-readable  form,  (2)  identifying  any  invalid  or 
inconsistent  values,  and  (3)  correcting  or  flagging  as  unresolvable  those 
values  identified  as  invalid  or  inconsistent.     Chapter  6.4  describes  these 
procedures. 

ETS  Quality  Control  procedures  were  used  to  assess  the  accuracy  of  the 
data  transcription  and  editing  operations.     Chapter  6.5  discusses  the  quality 
control  procedures  used  in  NAEP  data  processing  and  provides  a  summary  of  the 
likely  error  rates. 

Storage  of  Materials  refers  to  the  final  disposition  of  NAEP  printed 
materials  after  processing  had  been  completed.     Chapter  6.1  discusses 
materials  storage. 
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Database  Creation 


The  right-hand  side  of  Figure  6-1  depicts  the  evolution  of  the 
integrated  NAEP  database  from  the  transcribed  data  to  the  final  database, 
available  to  external  users  vi^   the  public -use  data  tapes.    Chapter  6.6 
describes  the  processes  throu3h  which  the  database  '^volved. 

Hie  remainder  of  this  section  contains  a  brief  description  of  each 
process  involved  in  database  creation  as  shown  on  the  figure.  Each 
description  also  refers  the  reader  to  the  sec;  .on(s)  or  chapter(s)  in  which 
the  process  is  discussed  in  detail. 

Sajnple  of  Schools  refc  rs  to  the  process  performed  by  Westat  to  select 
the  schools  to  be  included  in  the  assessment.  This  process  is  di&cussed  in 
Chapter  3. 

Data  Files  reffrs  to  (1)  the  data  files  created  by  the  ETS/NAEP  data 
transcription,  editing,  and  resolution  systems  and  (2)  the  lab^^^^^g  files 
(discussed  in  Chspttr  o.6)  that  contain  descripti\re  information  on  e\ery  item 
used  in  NAEP. 

Extract  is  the  process  (discussed  in  section  6.6.1)  that  created  data 
files  containing  specific  demographic  data  fields  extracted  from  the  ETS/NAEP 
data  files.    These  files  were  required  by  Westat  to  derive  sampling  weights. 

Sample  Weights  Derivation  was  performed  by  Westat  and  is  discussed  in 
Chapter  8.     This  process  produced  comput^ir  tape  files  containing  sampling 
weights  for  every  student  and  school  assessed  by  NAEP. 

Merge  refers  to  the  final  integration  of  NAEP  dat.    files  into  the  NAEP 
database.     This  process,  discussed  In  section  6.6.2,  merged  the  NAEP  data 
C'les,  labeling  files,  and  sampling  weights  into  one  database. 

*  AEP  Database  is  the  final,  integrated  W.EV  database  that  contains  all 
1988  NAEP  data  and  is  made  available  to  external  users  via  the  public -use  data 
tapes.    The  structure  of  the  internal  NAEP  database  is  di&ciissed  in  Chapter 
6.6;  the  '^vblic-use  data  tapes,  which  contain  all  of  the  nonconfidential  data 
fields  from  the  internal  database,  are  discussed  in  Chapter  6.7. 


6.0.4  NAEP  DATABASE:     DESCRIPTION  AND  HISTORY 

The  1988  NAEP  data  collection  resultec  in  seven  classes  of  data  file 
(student,  school,  teacher,  excluded  student,  sampling  weight,  item 
information,  and  derived  variable  files).    These  files  are  operationally 
merged  into  an  integrated  database  that  efficiently  lii.hs  student,  school,  and 
teacher  data  in  ways  that  permit  simultaneous  analysis.    The  database  system 
can  operationally  aggregate  data  while  avoiding  the  necessity  of  creating 
interi  diate  files. 

The  structure  and  internal  data  format  of  the  1988  NAEP  dat^ibase  is  a 
continuation  of  the  integrated  design  originally  develop^^d  by  ETS  in  1983. 
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Structure  of  the  NAEP  Database 

ETS  has  placed  all  NAEP  infcrnsation  from  the  1970-1988  assessments  into 
three  data  systems  that  make  up  the  complete  NAEP  database: 

The  Item  information  database  contains  all  of  the  descriptive, 
processing,  and  usage  information  for  every  item  developed  and  administered 
for  NAEP.     This  database  functions  as  a  resource  for  test  development 
activities,  data  system  control  operations,  and  item  linkage  to  past 
assessments.    A  complete  description  of  the  contents,  documentation,  and  uje 
of  this  database  is  found  in  A  Guide  to  the  NAEP  Item  Information  Database 
(Rogers,  Barone,  &  Kline,  1989)  and  A  Primer  for  the  NAEP  Item  Information 
Database  (Rogers,  Kline,  Barone,  Mychajlowycz,  &  Forer,  1989). 

The  restricted'Use  data  files  contain  all  NAEP  respondent  data, 
including  "secure"  files.    These  files  function  as  NAEP  data  archives  for 
responses  from  students,  teachers,  and  school  administrators  from  the  booklets 
and  questionnaires  used  in  NAEP  from  1970  to  1988.    The  contents, 
documentation,  anC  use  of  these  files  for  trend  analyses  are  described  in  A 
Guide  to  the  NAJEP  Restricted-use  Data  Files  (Rogers,  Barone,  &  Kline,  1989). 

The  publiC'Use  data  tapes  contain  all  nonconfidential  respondent  and 
item  information  for  public  dissemination.    Their  contents  and  use  are 
documented  in  The  1988  NAEP  Fablic-use  Data  Tapes  Version  2.0  User  Guide 
(Rogers,  Kline,  Johnson,  Mislevy,  &  Rust,  1990). 


These  three  database  products,  developed  by  ETS,  are  especially  useful 
because 

•  They  are  portable  and  can  be  use-"  on  a  variety  of  hardware 
systems.    They  can  be  accessed  by  a  variety  of  software  systems, 
including  SAS  and  SPSS. 

•  They  are  in  a  "rectangular"  file  structure  that  is  easy  to 
understand  and  use.    This  data  structure  eliminates  the  need  for 
complex  data  retrieval  processes  from  dissimilar  file  formats. 

•  They  have  standardized  documentation ^  including  complete  data 
layouts  and  codebooks. 

•  They  are  supported  by  user  guides  that  have  been  published  by 
ETS/NAE?. 


Size  of  the  NAEP  Database 


The  complete  NAEP  databe^se  is  massive.     Spanning  the  years  1970  through 
1988,  NAEP  has: 


These  data  are  maintained  as^  part  of  the  NAEP  item  information  or 
restricted-use  data  files.    Except  for  a  small  percentage  of  personal  or 
secure  item  information,  all  data  are  made  available  to  the  public  on  the 
public-use  data  tapes. 

While  the  selection  of  data  from  a  given  subject  area  or  assessment  year 
for  analysis  and  reporting  is  straightfoi  d  (documentation,  user  guides,  and 
data-variable  codebooks  are  available  for  t»very  NAEP  data  file),  the  following 
table  puts  the  toC:al  s*^e  of  the  NAEP  database  into  perspective. 


collected  data  on  the  performance  of  over  1.6  million  students  and 
young  adults,  and  background  information  about  their  teachers  and 
schools; 


collected  data  on  over  10,000  cognitive,  background,  and  attitude 
items  in  14  subject  areas  and  on  several  special  topics;  and 


o 


derived  over  3,000  variables  (composites,  transformations,  scale 
scores,  sampling  weights)  and  added  them  to  the  data  files. 


Approximate  Size 


NAEP  Database 


1970-1986 


1988 


Item  Information  Database 
Restricted-use  Data  Files 
Public-Use  Data  Files 


81,000,000  bytes 
2,000,000,000  bytes 
2,000,000,000  bytes 


4,000,000  bytes 
100,000,000  bytes 
100,000,000  bytes 
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Chapter  6.1 
PROCESSING  ASSESSMENT  HATERIALS 


Alfred  M.  Rogers  and  Norma  A.  Norris 


Educational  Testing  Service 


This  chapter  describes  t .  ^-  procedures  through  which  NAEP  instruments, 
schedules,  and  worksheets  were  received  at  ETS,  and  the  methods  used  in  the 
subsequent  scoring,  scanning,  loading,  editing,  and  r** solution  of  NAEP  data 


6.1.1  RECEIPT  OF  MATERIALS 

At  the  completion  of  the  assessment  administration  in  each  school,  it 
was  the  responsibility  of  the  Westat  district  supervisor  to  complete  and  mail 
a  postcard  to  ETS  containing  the  assessed  school  identification,  the  number  of 
boxes  shipped,  and  the  mode  of  shipment.    The  receipt  of  this  card  at  ETS 
alerted  staff  to  expect  arrival  of  the  shipment  within  seven  working  days.  If 
after  seven  days  the  shipment  had  not  arrived,  ETS  staff  were  directed  to 
notify  Westat,  who  in  turn  would  initiate  a  trace  of  the  shipment. 

The  shipment  from  each  school  contained  the  school  worksheet; 
administration  schedule;  questionnaire  roster;  school,  teacher,  and  excluded 
student  questionnaires;  and  assessment  booklets,  bundled  by  session.  The 
format  and  content  of  these  instruments  are  documented  in  the  Westat  Field 
Administration  Report.    The  following  discussion  of  check- in  procedures 
presumes  an  understanding  of  the  information  contained  in  and  the 
interrelationships  among  these  instruments. 

The  school  worksheet  contained  summary  counts  of  the  booklets  used  in 
all  assessment  sessions  in  each  school.    The  booklets  used  withir  each  session 
were  counted  and  checked  against  the  count  written  on  the  school  worksheet. 
All  discrepancies  in  the  counts  were  referred  to  the  administration  schedules 
for  resolution.     The  booklet  numbers  from  the  bundle  in  question  were  compared 
against  the  listing  of  booklet  numbers  on  the  schedule.     If  the  discrepancy 
could  not  be  resolved  by  this  process,  Westat  was  notified,  and  they  in  turn 
contacted  the  appropriate  district  supervisor  for  resolution  of  the 
discrepancy. 

'  Two  identification  codes,  the  session  code  and  the  batch  code,  were  then 
assigned  to  each  column  on  the  worksheet  and  to  the  corresponding  bundle  of 
booklets.  The  two-digit  session  code  distinguished  main  sessions  from  bridge 
sessions  and  regular  sessions  from  makeup  sessions. 

The  use  of  a  batch  identification  code  was  necessitated  by  the  up>e  of 
machine -scannable  documents  in  this  assessment.    A  preprinted,  scannable 
header  sheet  was  attached  to  each  bundle  of  student  booklets  to  be  used  to 
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identify  the  bundle  through  all  subsequent  scoring,  scanning,  entry,  and 
resolution  processing.    This  batch  header  sheet  was  pregridded  with  a  unique 
four-digit  sequence  code.     As  each  header  sheet  was  drawn  from  the  pile,  it 
was  gridded  with  the  age  group  code,  the  school  and  session  codes,  the  current 
batching  date,  and  the  nximber  of  booklets  to  be  processed.    The  age  group  code 
was  either  "N,"  "T,"  or  "S    corresponding  to  the  9-,  13-,  or  17-year-old 
cohorts.    The  batch  identification  code,  which  consisted  of  this  age  group 
code  and  the  sequence  number  from  the  header  sheet,  was  then  recordea  at  the 
bottom  of  the  session  information  on  the  school  worksheet. 

From  the  aspect  of  materials  processing,  a  significant  difference  of  the 
1988  assessment  from  the  1986  assessment  was  the  introduction  of  student 
instruments  with  removable,  machine -scannable  answer  sheets.    For  the 
grade  8/age  13  and  grade  12/age  17  cohorts  these  instruments  were  spiraled 
with  the  machine -scannable  inc^-^uments  whose  format  was  used  in  the  1986 
assessment  (the  grade  4/^5^  ^  cohort  were  administered  only  these 
machine -scannable  instrximents)  .    The  bundles  of  student  materials  for  each 
spiral  session  for  these  older  cohorts  thus  contained  a  mixture  of  scanv_-Lle 
booklets  and  scannable  answer  sheets. 

The  scanning  machine  program,  which  had  been  written  for  the  1986 
assessment,  was  used  again  to  scan  the  booklets;  a  new  program  was  written  for 
scanning  the  answer  sheets.    Since  these  programs  could  not  be  run 
concurrently,  the  bundles  of  instruments  had  to  be  split  into  separate  batches 
of  booklets  and  answer  sheets,  and  a  batch  header  sheet  assigned  to  each.  In 
order  to  maintain  the  integrity  of  each  session  throughout  the  processing 
phase,  the  same  batch  identification  number  had  to  be  assigned  to  each  pair  of 
booklets  and  answer  sheet  bundles  from  the  same  session. 

The  teacher  questionnaires  and  the  excluded  student  questionnaires  were 
then  counted  and  compared  against  the  questionnaire  roster.    All  discrepancies 
in  the  teacher  ou<jstionnaires  and  the  excluded  student  questionnaire  courts 
were  referred  to  Westat  and  again,  in  turn,  to  the  district  supervisor  for 
resolution.    As  the  field  administration  procedures  permitted  a  separate 
shipment  of  these  questionnaires,  the  questionnaire  roster  listed 
questionnaires  not  included  in  the  shipment,  alerting  the  receiving  staff  to 
expect  a  later  shipment. 

If  the  supervisor  was  unable  to  collect  the  questionnaires  on  the  day  of 
the  assessment,  a  pre- addressed  envelope  was  left  at  the  school  so  that  the 
school  coordinator  could  mail  the  questionnaires  directly  to  ETS.    There  was 
no  other  follow-up  activity  to  obtain  uncollected  questionnaires  from  school 
personnel;  efforts  to  encourage  school  cooperation  were  focused  primarily  or 
student  assessment  activities. 

When  all  of  the  student- related  materials  for  a  school  had  been  received 
and  checked  in,  the  administration  schedules,  school  worksheet,  assessment 
booklets,  and  questionnaires  were  forwarded  to  the  data  operations  coordinator 
for  transcription  processing.    The  operations  coordinator  separated  these 
materials  according  to  the  appropriate  data  entry  procedures:  the 
administration  schedules,  the  school  worksheet  and  the  teacher  and  school 
questionnaires  were  sent  directly  to  data  entry  systems;  the  excluded  student 
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questionnaires  were  accumulated  and  shipped  in  batches  to  the  optical  scanning 
area;  and  the  assessment  session  bundles  were  forwarded  to  the  professional 
scoring  area. 

The  absentee  data  from  the  administration  schedules  and  the  school 
worksheet  data  were  entered  into  the  data  entry  system  on  a  daily  basis.  The 
teacher  and  school  questionnaires  were  batched  and  held  for  data  entry  until 
scheduling  permitted. 


6*  1,2    SCHOOL  WORKSHEETS 

Each  column  of  the  school  worksheet  contained  information  pertaining  to 
the  administration  activity  of  each  session  within  a  school.    This  information 
included  the  date,  time,  and  location  of  the  adniinistration,  the  exercise 
administrator  code,  and  the  counts  of  -.he  students  who  were  sampled,  those  who 
were  absent,  and  those  who  v:ere  assebi>ed.    Additionally,  each  column  contained 
a  session  code  and  batch  identification  code  that  were  recorded  by  receipt 
processing  staff.    This  information  was  entered  into  the  system  by  selecting 
the  first  option  on  the  main  data  entry  menu  (Figure  6.1-1). 


Figure  6.1-1 
Main  Menu  for  the  NAEP  Data  Entry  System 


N  A 

EP      ENTRY      SYSTEM  MENU 

OPTION: 

1 

School  Worksheet  Entry 

2 

Student  Data  Entry /Verification/Resolution 

3 

Questionnaire  Data  Entry /Verification/Resolution 

A 

Absentee  Data  Entry 

5 

Questionnaire  Roster  Entry 

X 

Quit 

Enter  Option  Cnde 
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The  worksheet  entry  pr'^gram  received  its  input  through  two  entry 
screens.    The  first  entry  scieen  (Figure  6.1-2)  requested  school-level 
information,  namely,  the  PSU  and  school  codes  and  the  total  number  of 
assessment  sessions  that  were  conducted  in  that  school.    This  count  was 
further  broken  down  into  the  four  types  of  session  administration;  regular 
spiral,  makeup  spiral,  regular  bridge,  and  makeup  bridge.    The  program  would 
then  display  the  second  entry  screen  (Figure  6.1-3)  once  for  each  session, 
requesting  the  session-level  information.    When  all  sessions  for  a  school  had 
been  entered,  the  program  would  redisplay  the  first  entry  screen,  ready  to 
process  the  next  worksheet.    The  operator  could  either  enter  new  information 
or  press  ENTER  to  return  to  the  main  menu. 

The  assessment  session  was  the  primary  unit  at  which  the  entry  system 
controlled  the  processing  of  student  data  and  maintained  statistics  on  data 
entry  activity.    A  separate  tracking  file  was  established  for  this  purpose, 
each  record  of  which  contained  all  control  and  reporting  information  for  one 
sessi — .    The  entry  of  the  school  worksheet  information  thus  generated  a  new 
record  on  the  tracking  file  for  each  session,  setting  initial  values  for  those 
parameters  that  would  control  entry  processing  and  record  entry  events. 

The  operations  coordinator  was  provided  with  procedures  for  periodically 
monitoring  and  reporting  activity  on  the  data  entry  system.    These  procedures 
compared  the  counts  of  booklets  processed  at  each  stage  with  the  initial 
counts  from  the  worksheet,  and  flagged  discrepancies.    This,  in  turn,  alerted 
the  coordinator  to  possible  missing  or  extra  booklets.    If  the  school 
worksheet  information  was  determined  to  be  in  error,  the  operations 
coordinator  had  the  facility  to  correct  the  tracking  file  data  to  prevent 
reappearance  of  the  discrepancies  in  the  activity  report. 

The  school  worksheets  were  retained  by  the  operations  coordinator  in 
anticipation  of  later  queries,  since  they  could  be  stored  conveniently  and 
referenced  easily. 


6.1.3    ADMINISTRATION  SCHEDULES 

The  administration  schedules  contain  the  demographic  characteristics  of 
the  students  selected  for  the  assessment.     This  information,  which  included 
the  gender,  race/ethnicity,  grade,  £,nd  birth  date  of  the  sampled  students,  was 
used  by  Westat  in  the  derivation  of  sampling  weights.    The  booklet  numbers  of 
the  students  who  participated  were  transferred  to  the  schedule  at  the  time  of 
the  assessment,  and  the  demographic  information  was  in  turn  transferred  to  the 
front  covers  of  the  booklets  after  the  assessment. 

The  demographics  of  the  students  who  were  sampled  but  did  not 
participate  in  the  assessment  (exclusions  and  absentees)  were  used  to  adjust 
the  sampling  weights  of  the  students  who  did  participate.    The  excluded 
student  information  could  be  obtained  from  the  excluded  student  questionnaire 
data,  but  the  information  on  absentees  could  be  found  only  on  the 


ERLC 


4 

f 

112 


li>.8 


Figure  6.1-2 
First  School  Worksheet  Data  Entry  Screen 


SCHOOL  WORKSHEET 


PSU  #: 
SCHOOL  #: 


TOTAL  NUMBER  OF  SESSIONS: 

NUMBER  OF  SPIRAL  SESSIONS  (0-10): 
NUMBER  OF  MAKEUP  SPIRAL  SESSIONS  (0-10): 
NUMBER  OF  ORIGINAL  TAPE  SESSIONS  (0,1,2): 
NUMBER  OF  MAKEUP  TAPE  SESSIONS  (0,1,2): 

I  


Figure  6.1-3 
Second  School  Worksheet  Data  Entry  Screen 


SCHOOL  WORKSHEET 

PSU  #:   

SCHOOL  #:   

TAPE/SESSION  #:   

DATE:  _y __/_ 

TIME:   :_ 

EA'S  INITIALS:   

EA'S  ID:   

#  TO  BE  ASSESSED:   

#  ASSESSED:   

#  ABSENT:   

BATCH  NUMBER: 

1 
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acinirdstration  schedules.  It  was  therefore  necessary  to  transcribe  the 
J.nfr/rmation  for  these  absentees  to  computer -readable  media  and  combine  it  with 
the  assessed  and  excluded  student  data. 

Tlie  absentee  data  were  entered  into  the  system  by  selecting  the  fourth 
w^ption  on  the  main  data  entry  menu.    The  system  then  presented  a  screen  for 
the  entry  of  all  absentee  information  for  a  single  session  (Figure  6.1-4). 
The  program  fir  »t  requested  entry  of  the  batch  identification  nximber  and  the 
irSU,  school  and  session  codes.    The  batch  code  was  used  to  locate  the 
corresponding  session  record  in  the  tracking  file  and  the  remaining  codes  were 
checked  for  correctness  against  the  corresponding  fields  in  the  tracking 
record. 

Figure  6.1-4 
Absentee  Data  Entry  Screen 


NAEP     ABSENTEE  ENTRY 
P*TCH  ID:    iSU:    SaXX)L:    SESSION:  _ 


GRADE  SEX      B.D.        GRADE  SEX     B.D.        GRADE  SEX     B.D.        GRADE  SEX     B.D.        GRADE  SEX  B.D. 


The  remainder  of  the  screen  could  accommodate  data  entry  foi:  75  absent 
students.    Only  three  data  fields  were  required  for  each  absentee:  grnde, 
gender  and  birth  date.    These  data  were  ultimately  used  by  Wcstat  to  ^^djust 
the  sample  weights.    As  each  field  was  entered,  the  program  checked  for 
appropriateness  of  range  according  to  the  age  cohort  ana  :,ession  t.ype.    At  the 
completion  of  data  entry  for  an  assessment  session,  the  operate",  pressed  ENTER 
and  the  program  would  present  a  fresh  screen  for  the  entry  of  another 
session's  complement  of  absentee  data.    The  operator  could  then  tithei  enter 
another  batch  identification  code  or  press  ENTER  again  to  return  to  tbc  ir.cxla 


menu. 


If  the  operator  entered  a  session  cod^  for  which  absentee  data  had 
already  been  entered,  the  program  would  display  the  data  for  all  absentees  in 
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the  record  for  that  session  at  that  time.    The  operator  then  had  the  facility 
to  correct,  delete,  or  add  new  data.    When  the  ENTER  key  was  pressed,  the  data 
displayed  on  the  screen  were  transcribed  to  the  current  session  record  and  the 
record  was  rewritten  to  the:  ab^^entee  data  file. 

The  assessment  schedulx.£>  were  retained  by  the  operations  coordinator  in 
anticipation  of  future  questions  about  and  references  to  the  sample.    As  the 
schedules  for  all  three  grade/age  assessments  could  be  contained  in  three 
storage  boxes,  their  retention  proved  to  be  the  most  efficient  and  compact 
means  of  referring  to  the  relevant  raw  data. 


6.1.4    STUDENT  ASSESfmSNT  INSTRUMENTS 

The  studeat  assessment  booklets  w^re  forwarded  directly  to  the  scoring 
area  as  the  coraplete  set  of  materials  was  received  from  each  school.  The 
booklets  and  answer  sheets  were  batched  separately  for  each  session,  with  a 
batch  header  sheet  attached  to  the  top  of  each  bundle.     This  preprinted, 
scannable  sheet  contained  the  PSU,  school,  and  session  codes,  and  a  unique 
batch  identification  code  to  identify  each  session.    The  header  sheets  were 
retained  with  the  batcbc5  throughout  entry  processing. 


6.1.4.1    Professional  Scoring 

The  batches  of  stuuent  booklets  and  answer  sheets  were  sent  from  the 
receipt  processing  area  to  the  scoring  area  where  the  open-ended  reading, 
writing,  mathematics,  and  civics  items  were  read  and  scored  by  trained 
readers.    The  procedures  and  guidelines  followed  in  ocoring  these  items  are 
more  fully  described  in  Chapter  6.2. 

Each  open-ended  item  was  provided  with  a  set  of  scannable  ovals  to  be 
filled  in  by  the  reader.     The  ovals  were  usually  at  the  bottom  of  the  page  on 
which  the  item  was  printed  to  avoid  distracting  or  confusing  the  student. 
All  open-ended  items  were  provided  with  an  extra  set  of  cvals  to  permit 
secondary  scoring  of  the  primary  trait  scores  for  interrat^r  reliability 
analysis.     Several  of  the  reading  and  writing  items  that  were  to  be  evaluated 
for  secondary  traits  had  an  additional  set  of  ovals  for  each  secondary  trait 
score . 

The  primary  reader  would  examine  each  booklet  in  a  batch  and  determine 
if  it  contained  any  open-ended  items.    If  so,  the  reader  wrote  in  his  or  her 
identification  code  and  gridded  in  the  first  column  jf  ovals  in  the  reader 
identification  area  on  the  inside  frotit  page.    The  leader  would  then  locate 
and  read  each  of  the  open-ended  items  for  th«t  booklet  and  grid  the  first 
primary  trait  score  and  all  secondary  trait  scores  into  the  appropriate  ovals. 
On  every  fifth  booklet  read,  the  reader  would  place  a  piece  of  tape  over  each 
set  of  primary  trait  score  ovals  and  designate  this  book  for  secondary 
scoring.    The  sampling  rate  of  one  in  five  assured  a  20  percent  rate  of 
secondary  scoring,  and  the  tape  was  a  device  to  avoid  influencing  the 
secondary  reader.    The  completed  booklets  were  stacked  in  the  same  order  in 
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which  they  were  received  and  the  completed  batch  was  placed  it.  a  designated 
area. 

The  secondary  reader  selected  the  designated  booklets  from  each  batch 
and  entered  his  or  her  identification  code  into  the  second  column  of  the 
reader  .identification  r.rea.    The  reader  then  located  the  items  with  concealed 
primary  trait  scores,  read  and  scoreJ  them,  and  removed  the  pieces  of  tape. 
The  completed  batch  was  placed  in  a  second  holding  area,  whence  it  would  be 
forwarded  to  the  scanning  processing  area. 


6,1.4.2  Scanning 

The  heart  of  the  scanning  proi^^ss  was  a  programmable  compu^-ing  machine 
that  could  "read"  the  pencil  marks  from  b'  'i  sides  of  a  cheet  of  paper  at  a 
very  high  speed,  convert  those  readings  to     response"  codes,  and  transcribe 
those  codes  to  a  magnetic  computer  tape  iu      specified  format.     This  section 
addresses  the  functions  that  constitute  the  oody  of  the  scanning  process:  the 
preparation  of  the  scannable  materials,  the  operation  of  the  scanning  machine, 
and  the  activity  of  the  scanning  machine  operator. 

Before  the  batches  of  scored  booklets  and  scored  answer  sheets  were  'ienc 
to  the  scanning  area,  \hey  were  grouped  by  age  cohort  and  placed  into 
"capsules"  that  were  chen  arranged  sequentially  on  "carts."    The  capsules  were 
cardboard  boxes  with  one  side  open  to  facilitate  access  by  the  scanning  and 
resolution  staff,  and  uhe  other  side  equipped  with  hangers  to  pe/mit  removal 
from  thte  carts.    The  carts  were  transportable,  two-sided  hanging  shelves  with 
sloping  sides  to  permit  the  capsules  to  hang  with  the  open  sides  out  while 
keeping  the  documents  in.     The  carts  were  shipped  to  the  scanning  area. 

The  first  step  in  the  scanning  process  was  to  separate  each  booklet  into 
its  component  pages  for  single-sheet  processing  by  the  scanner.     Each  booklet 
was  secured  by  three  staples  along  the  left  edge.     Each  answer  sheet  was  a 
large,  single  sheet  of  paper  folded  along  the  left  edge.     The  timing  marks  for 
the  scanner  were  also  printed  along  tnls  edge.    Two  special  machines  were  used 
to  cut  off  the  stapled  edge  without  di.maging  the  aiming  marks.     The  cutting 
machine  could  cut  three  or  four  booklecs  at  a  time  but  required  a  slower, 
manual  setup  process.    The  slitting  machine  was  more  automatic,  processing  one 
booklet  at  a  time,  but  was  less  precise  than  the  cutting  .Tiachine.  Careful 
handling  of  these  booklets  was  imperative  once  they  were  cut,  as  the  scanning 
program  depended  on  the  correct  sequencing  of  pages  within  each  booklet.  The 
cut  booklets  were  placec?  back  in  their  capsules  and  the  completed  cart  sent  to 
the  scanning  machine . 

The  scanning  machine  operator  first  determined  which  scanning  program  to 
be  used  according  to  the  age  cohoit  and  instrument  type  (booklet  or  answer 
sheet),  mounted  a  fresh  magnetic  tape  on  the  machine's  tape  drive,  and  started 
miming  the  program.    Scanning  was  initiated  by  placing  the  sheets  from  the 
first  capsule  into  the  input  hopper  of  the  scanning  de^'ice.     The  scanner  thon 
read  both  sides  of  each  sheet  and  placed  it  into  one  of  two  hoppers.     If  no 
errors  in  readability  or  sequencing  were  datectcid,  the  sheet  went  into  the 
output  hopper  and  the  next  sheet  was  read  from  the  input  hopper.    If  an  error 
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was  indicated,  the  sheet  was  diverted  *.ito  the  shunt  hopper,  the  program  wrote 
a  message  to  the  operator's  console,  a..d  the  scanner  stopped  processing  while 
the  operator  took  appropriate  action. 

Each  page  of  every  booklet  and  answer  sheet  had  been  printed  with  a  set 
of  identification  marks  next  to  the  timing  marks.    The  front  cover  of  »ach 
booklet  \%\thin  a  numbered  type  was  assigned  a  unique  set  of  these  marks,  and 
the  pages  within  each  item  block  type  were  sijiilarly  identified  by  block  code 
and  sequence  number.    As  the  scanner  read  a  cover  sheet,  the  program 
identifier  the  booklet  number  and  referred  to  an  internal  table  to  determine 
which  blocks  should  follow  and  which  page  formats  should  appear  within  each 
block. 

If  the  program  indicated  a  page  sequence  error,  the  operator  instructed 
the  program  to  treat  the  page  as  missing  and  placed  the  shunted  page  into  the 
input  hopper  to  be  read  again.     If  a  page  within  a  block  was  unreadable,  che 
operator  instructed  the  program  to  treat  it  as  missing  and  placed  the  sheet 
perpendicularly  on  top  of  the  output  stack.     If  a  block  sequence  error  or 
unreadable  booklet  cover  was  indicated,  the  operator  instructed  the  program  tu 
insert  a  dummy  record  and  removed  the  remaining  pages  of  that  booklet  and 
placed  them  perpendicularly  on  top  of  the  output  stack. 

As  the  scanning  program  completed  scanning  each  batch,  the  batch  was 
removed  £tom  the  output  hopper  and  placed  back  in  its  capsule.    The  next  batch 
was  taken  from  its  capsule  and  placed  into  the  input  hopper  and  the  machine 
resumed  processing.    When  the  machine  had  completed  processing  the  last  batch, 
the  operator  terminated  the  program,  dismounted  the  **ape,  and  removed  the 
listings  from  the  printer. 

The  output  data  tapes  were  forwarded  to  the  VAX  computer  area  for 
locding  processing.    The  scanned  documents  were  returned  in  their  original 
cat'tons  to  the  resolution  processing  area. 


6.1.4.3  Loac^^ng 

The  scanning  tapes  were  recei      ^  and  checked  in  by  an  operator  at  the 
VAX  computer  area*      The  operations  coordinator,  having  received  notification 
of  the  tape's  transmittal,  initiated  the  data  eutry  management  procedure  on 
the  VAX  computer  from  which  the  "Load  Scanning  Tapes"  option  was  selected 
(Figure  6.1-5).    A  second  menu  provided  the  super^'isor  with  the  choice  of 
loading  the  scannable  booklets,  answer  sheets,  or  excluded  student 
questionnaires  (Figure  6  ''-G). 

The  program's  first  input  request  was  the  tape  number,  a  six-digit  code 
printed  on  an  external  label  on  the  tapt  and  coded  internally  by  the  scanning 
program.    Ttie  VAJC  operator  then  mounted  that  tape  on  the  tape  drive  and  put 
the  drive  online,  which  logically  connected  the  tape  to  the  program.  The 
program  checked  that  the  right  tape  number  and  type  had  been  mounted  and 
proceeded  with  the  loading  process.    As  it  processed  the  tape,  the  program 
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Figure  6.1-5 

Main  L'enu  for  Management  of  the  NAE?  Da^a  Entry  System 


N  A  E  P 

DATA      ENTRY  MANAGEMENT 

OPTION: 

1 

Tracking  &  Data  File  Management 

2 

Status  Reports 

3 

Questionnaire  Audit 

4 

Load  Scanning  Tapes 

5 

Quality  Control 

6 

Spool  Data  for  Final  Edits 

X 

Quit 

Enter  Option  Code: 

Figure  6.1-6 

Menu  for  Loading  of  D^ta  Tapes  into  NAEP  Data  Entry  System 

NAEP  DATA 

TAPE      LOADING  FUNCTIONS 

OPTION: 

1 

load  Tape  of  Scannable  Booklets 

2 

Load  Tape  of  Scannable  Answer  Sheets 

3 

Load  Tape  of  Excluded  Student  Questionnaires 

X 

Quit 

Enter  Option  Code: 
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prlntsd  the  batch  code  and  record  count  for  each  batch  to  the  operator's 
terminal  to  assure  the  operator  that  the  program  was  running  prc-erly  When 
the  program  reached  the  end  of  the  tape  file,  it  printed  out  three  listings, 
rewound  and  dismounted  the  tape,  and  returned  to  the  main  menu.    The  three 
listings  consisted  of  an  error  log,  a  batch  listing,  and  an  audit  listing. 

The  error  log  was  a  running  commentary  and  summary  uf  the  processing  of 
the  tape.     Each  log  was  identified  with  Lha  tape  number,  file  name,  and  date 
of  the  loading  run.     The  start  of  each  batch  was  recorded  with  the  batch 
number  and  its  corresponding  school  and  session  codes.    Any  disagreement 
between  these  codes  and  those  entered  from  ti.e  school  worksheet  was  recorded 
at  this  point.    Any  booklets  that  did  not  belong  to  the  session  type  (e.g., 
bridge  booklets  ir  a  spiral  session)  were  also  listed  here       well  as  all 
unscannable  booklets.    At  the  end  of  processing  each  batch,  the  program 
printed  the  number  of  scannable  and  unscannable  booklets  it  had  counted  in 
that  batch. 

The  batch  listing  reported  the  information  from  the  front  cover  fields 
of  each  booklet  within  each  batch.    This  listing  could  be  checked  against  the 
administration  schedules  for  discrepant  or  missing  information. 

The  audit  listing  identified  the  data  problems  found  within  each  batch. 
Each  data  anomaly  was  identifier  by  the  batch  sequence  number,  booklet  numb^^.r  , 
section,  and  item  number  to  facilitate  location  of  the  data  in  the  actual 
instruments  by  resolution  staff. 

The  printed  output  was  forwarded  to  the  resolution  area  to  be  joined 
with  the  scanned  materials.    The  tape  was  retained  in  the  VAX  computer  area. 


6.1.4.4    Resolution  Processing 

The  error  log  and  batch  listing  were  retained  by  the  operations 
coordinator.     The  audit  listings  were  separated  by  batch  number  and  mat*-  .d 
with  the  appropriate  scanned  materials.     If  the  error  log  indicated  any 
unscannable  booklets  within  a  batch,  they  were  identified  and  extracted  from 
the  bundle  and  manually  entered  and  verified  through  the  data  entry  system. 
Upon  completion  of  verification  processing,  the  system  produced  an  updated 
audit  listing  to  replace  the  one  output  from  the  load  process. 

Resolution  processing  was  not  permitted  to  proceed  unti"!  all  materials 
from  a  session  had  been  scanned,  loaded  and  received,  after  wh^^h  the  separate 
booklet  and  answer  sheet  batches  from  a  ses.,ion  were  matched,  combined,  and 
forwarded  to  resolutions  staff. 

Staff  assigned  to  resolution  processing  reviewed  the  audit  listing, 
checked  the  actual  responses  in  the  documents  wherever  asterisks  or  quesLiou 
marks  were  indicated,  determined  the  appropriate  value(s)  to  be  coded  in  tht 
data  file,  and  wrote  these  new  codes  on  the  audit  listing.  The  asterisks 
indicated  multiple  gridding  of  a  single- response  item,  question  marks  flagged 
critical  fields  from  the  front  cover,  such  as  gender  or  birth  date,  that  were 
incorrectly  gridded,  and  fields  from  unscannable  pages. 
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Access  to  the  student  data  for  entry,  verification,  or  resolution 
processing  was  gained  through  the  second  option  on  the  main  data  entry  menu 
The  first  screen  (Figure  6.1-7)  requested  the  identification  number  of  the 
batch  to  be  processed;  the  PSU,  school,  and  session  codes  as  a  secondary  check 
on  the  batch  code;  and  a  code  for  the  processing  mode:  entry,  verification,  or 
resolution.    The  second  entry  screen  (Figure  6.1-8)  prompted  for  input  of  the 
batch  serial  number  and  for  the  student  ID  number  as  a  secondary  check. 

The  processing  mode  was  a  program  st5*te  that  determined  how  much  data 
the  terminal  operator  would  see,  how  to  process  data  entered  by  the  operator, 
and  the  management  of  data  within  the  program.    The  entry  mode  permitted  the 
creation  of  a  new  data  record  by  the  operator.    The  verification  mode  presumea 
the  existence  of  a  data  record,  permitted  the  "creation"  of  a  second  daJ-a 
record,  and  performed  a  field-by-field  comparison  of  thr  -wo  records,  alerting; 
the  operator  of  data  disagreements.    The  resolution  mode  displayed  the  entire 
contents  of  a  data  record  and  interpreted  any  data  en-ered  b>  the  operator  as 
a  correction  of  a  data  field. 

The  resolution  mode  of  the  entry  system  permitted  the  operator  to  read 
verified  or  loaded  data  records,  display  the  field  values,  and  make 
corrections  to  individual  fields.    A  change  in  any  data  field  under  resolution 
mode  also  generated  a  record  for  the  audit  file,  and  the  program  produced  an 
updated  audit  listing  at  the  completion  of  resolution  processing  for  each 
batch.    There  was  no  limit  to  the  number  of  times  a  session  or  data  record 
could  be  processed  under  resolution.    On  completion  of  resolution  processing, 
each  session  bundle  was  stored  in  a  labeled  box  and  held  for  final  editing  and 
quality  control  processing. 

If  the  program  was  in  the  entry  mode  and  no  data  record  for  the  booklet 
could  be  found,  the  program  would  prepare  to  create  a  new  record  and  request 
entiy  of  the  booklet  cover  data.     If  in  verification  mode  and  the  data  record 
had  not  been  already  verified,  the  program  would  request  re-entry  of  the  cover 
data  and  compare  them  against  the  data  record.     If  in  resolution  mode  and  the 
data  record  had  been  through  verification  or  loading  processing,  all  data 
fields  were  displayed  and  the  operator  could  either  modify  these  fields  or 
advance  to  the  rest  of  the  entry  screens  for  that  booklet. 

A  final  validation  was  performed  when  the  data  entry  work  files  were 
iL::*ged  and  copied,  c:  spooled,  onto  a  master  student  data  file.    This  spooling 
program  checked  every  data  field  of  every  student  record  for  ov.t- of -range 
values  and  question  marks.    A  listing  similar  to  the  audit  listings  for  each 
session  was  produced,  which  the  resolution  staff  then  used  to  identify  and 
correct  the  remaining  data  anomalies. 

The  quality  control  process  first  selected  a  random  sample  of  each 
booklet  type  from  the  master  student  file,  identifying  them  by  batch  and 
sequence  nximber.    The  designated  booklets  were  located,  extracted  from  their 
storage  boxes,  and  forwarded  to  the  quality  control  staff.    The  responses  in 
each  booklet  were  then  compared  with  their  coded  data  values  in  the  dtta  file 
On  completion  c"  ^uality  control  processing,  the  booklets  were  returned  to 
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Figure  6,1-7 
Student  Session  Data  Entry  Screen 
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Figure  6.1-8 
Student  Booklet  Cover  Data  Entry  Screen 
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their  boxes.  The  full  details  and  results  of  the  quality  control  process  are 
presented  in  Chapter  6.5. 

At  the  completion  of  resolution  processing  of  the  student  data,  the 
session  bundles  were  separated  by  booklet  or  answer  sheet  and  sorted  into 
individual  stacks  by  instrument  number.    These  stacks  were  then  placed  into 
storage  boxes,   identified  by  age  cohort  and  instrument  number,  and  shipp«id  to 
the  ETS  data  retention  area  for  long-term  storage. 


6.1.5  QUESTIONNAIRES 

The  questionnaire  instruments  were  separated  by  type  and  accumulated  by 
the  operations  coordinator  as  they  were  received  from  mail  processing.  The 
teacher  and  school  quest\onnaires  were  eventually  transcribed  through  the  NAEP 
data  entry  system,  but  on  a  lower  priority  basis  than  the  student  booklets. 
The  excluded  student  questionnaires    were  batched  and  sent  to  scannii  ^  at 
regular  intervals,  since  the  demographics  of  the  excluded  students  were  usci 
in  deriving  the  sampling  weights  of  the  assessed  students.     In  order  to  allow 
the  two  files  to  be  completed  at  the  same  time,  every  effort  was  made  to  keep 
the  process^.ng  rate  of  these  instruments  in  pace  with  the  student  data  entry. 

Processing  of  the  questionnaire  data  was  initiated  by  selecting  the 
third  option  on  the  data  entiry  menu.    The  first  entry  screen  (Figure  6.1-9) 
prompted  for  input  of  the  questionnaire  type,  age  group,  and  processing  mode. 
The  questionnaire  entry  programs  followed  the  same  model  as  the  student  entry 
program  with  the  absence  of  a  tracking  file  and  session  batching.  Entry, 
verification,  and  resolution  modes  were  available;  audit  reports  were 
initiated  by  the  operations  coordinator. 


Figure  6.1-9 


Primary  Menu  for  the  Entry  of  Questionnaire  Data 


NAEP  YEAR  19  QUESTIONNAIRE  MENU 


TYPE:  _  AGE:  _  MODE: 

1  SCHOOL  1    AGE  9  1  ENTRY 

2  TEACHER  2    AGE  13  2  VERIFICATION 

3  EXCLUDED  STUDENT  3    AGE  17  3  RESOLUTION 
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The  data  for  the  excluded  student  questionnaires  for  the  age  13  bridge 
sample  were  entered  through  the  data  entxy  system.    The  program  lor  entry  of 
the  exclu^ied  student  questionn'*ire  data  first  displayed  a  screen  for  entry  of 
the  front  cover  data.    The  operator  was  prompted  for  the  serial  number  of  the 
booklet  to  be  processed.    An  error  condition  occurred  when  either  a  record 
with  that  serial  number  was  found  under  entry  mode  or  when  no  record  was  found 
under  the  verification  or  resolution  mode.    In  either  case,  the  operator  was 
asked  to  verify  that  the  correct  number  had  been  entered.     If  the  problem 
persisted,  it  was  referred  to  the  operations  coordinator  for  resolution.  The 
remaining  cover  information,  including  PSU  and  school  code,  student  gender, 
race/ethnicity,  grade,  and  birth  date,  was  processed  acco^'ding  to  the  same 
criteria  as  were  the  data  from  the  student  booklet  covers.    The  program  then 
displayed  a  single  screen  for  processing  the  responses  within  the 
questionnaire.    When  the  operator  pressed  ENTER  to  terminate  processing  for 
that  booklet,  the  program  redisplayed  the  cover  entry  screen,  ready  to  process 
another  booklet.    A  blank  field  entered  in  the  serial  number  field  returned 
the  program  to  the  primary  menu. 

The  excluded  student  questionnaires  for  t-he  age  9  bridge,  age  17  bridge, 
and  all  main  samples  contained  detachable  machine -scannable  answer  sheets  for 
the  recording  of  identification  information  and  responses  to  the  questions. 
These  sheets  were  checked  for  correctness  and  completeness  of  identification 
information,  separated  from  their  questionnaires,  and  placed  on  one  cf  three 
piles,  according  to  age  cohort.  On  a  weekly  basis,  the  operations  coordinator 
would  grid  a  batch  header  sheet  for  each  pile  with  the  appropriate  age  cohort 
information,  place  it  on  the  appropriate  pile,  and  send  these  batches  to  the 
scanning  area. 

The  scanning  program  for  the  excluded  student  answer  sheets  was  executed 
once  for  each  batch,  creating  a  separate  output  tape  for  each  age  cohort.  At 
the  completion  of  scanning  processing,  the  tapes  were  sent  to  the  VAX  computer 
are^i  and  the  batches  sent  back  to  the  data  entry  area.    Upon  notification  of 
the  tape  serial  numbers,  the  operations  coordinator  started  the  data  entry 
management  procedure  on  the  VAX  computer,  selected  the  "Load  Scanning  Tapes" 
option,  and  chose  the  third  option  to  initiate  loading  of  the  excluded  student 
data. 

The  loading  program  for  the  excluded  student  data  performed  many  of  the 
same  functions  as  the  progra*^  for  loading  the  assessed  student  data:  checking 
the  demographic  information  for  appropriateness  to  age  cohort  and  sample, 
validating  the  questionnaire  responses  for  range,  ard  reformatting  the  output 
records  for  compatibility  with  the  data  entry  system.     The  entry  system 
maintained  a  single  excluded  student  data  file  and  p  single  audit  file  for 
each  age  cohort.    The  load  program  wrote  the  edited  Jata  records  to  the 
appropriate  data  file,  using  the  unique  book  serial  number  as  an  indexing  key 
for  insertion  into  the  file,  and  for  later  retrieval  by  the  resolution 
program.    The  program  also  wrote  the  audit  records  to  the  appropriate  audit 
file  for  each  age  cohort,  "appending"  them  to  the  end  of  the  file.     When  the 
program  completed  loading  the  data,  it  produced  an  audit  listing  of  data 
anomalies  found  in  that  batch. 
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Resolution  processing  started  by  comparing  the  anomalous  data  in  the 
audit  listing  with  the  mf.rked  responses  on  the  answer  sheets.    All  corrections 
we^e  recorded  on  the  listing,  which  was  given  to  a  data  entry  operator.  The 
operator  selected  the  third  option  from  the  main  menu  to  display  the 
questionnaire  data  entry  menu,  then  indicated  resolution  of  excluded  student 
questionnaire  data  for  the  appropriate  age  cohort.    After  entering  the 
identification  number  for  tl     reco  d  to  be  corrected,  the  operator  confirmed 
that  the  right  record  had  been  retrieved  and  made  the  corrections  to  the 
erroneous  data.    An  account  of  each  data  correction  was  written  to  che  audit 
file. 

The  operations  coordint.tor  had  the  ability  to  produce,  either  as  needed 
or  at  the  conclusion  of  all  resolution  processing  for  an  age  cohort,  a 
complete  audit  listing  for  any  questionnaire.     This  listing  was  organized  by 
transaction  within  data  field  within  identification  number,  to  facilitate 
visual  analysis  of  resolution  activity.    At  the  conclusion  of  the  f^xcluded 
student  data  processing  for  an  age  cohort,  a  special  summary  proj^j-am  was 
executed  that  listed  and  counted  the  processed  questionnaires  within  each 
school.    This  listing  was  compared  against  the  individual  questionnaire 
rosters  for  each  school  to  determine  if  any  shipments  were  still  incomplete. 
The  rosters  contained  enough  information  to  generate  "dununy"  records  for  each 
missing  questionnaire,  which  were  designated  with  a  special  code  for  use  by 
Westat  in  deriving  the  sample  weights.    These  records  were  added  to  the  file 
that  was  sent  to  Westat  along  with  the  absentee  and  assessed  student  data 
files. 

The  program  for  entry  of  the  teacher  questionnaire  data  first  displayed 
a  screen  for  entry  of  the  cover  information.    It  processed  the  i>erial  nximber 
in  the  same  fashion  as  did  the  entry  program  for  the  excluded  student 
questionnaire.     The  cover  information  included  only  the  PSU,  school,  and 
teacher  codes.    As  the  longest  questionnaire  instrximent,  the  teacher 
questionnaire  required  three  screens  for  entry  processing.    Completion  of 
processing  for  each  booklet  returned  the  program  to  the  covei  entry  screen, 
where  the  entry  of  a  blank  serial  number  returned  the  program  to  the  primary 
menu. 

The  program  for  entry  of  the  school  questionnaire  data  also  a carted  with 
a  display  ol  the  cover  entry  screen.    The  only  information  requested  for  this 
instrximent,  however,  was  the  PSU  and  school  code,  which  also  served  as  the 
booklet  identification  nun»ber.     Bec/u<:e  of  the  large  number  of  questions  in 
this  questionnaire,  entry  processing  required  two  screens.    Completion  of 
pre  .essing  for  each  booklet  returned  the  program  to  the  cover  entry  s^^een, 
vhere  the  entry  of  a  blank  PSU  and  school  code  returned  the  program  to  the 
primary  menu. 

After  all  questionnaires  had  been  received  and  processed  through  the 
encry  system,  a  validation  program  was  run  against  all  data  values  in  all 
records.    All  remaining  dat.i  errors  or  discrepancies  were  then  correct ''d  using 
the  resolution  mode  of  the  entry  system.    A  linal  audit  listing  was  generated, 
recording  all  entry  activities  for  each  questionnaire. 
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The  questionnaires  were  subjected  to  the  same  quality  control  procedures 
received  by  the  student  data.    The  details  of  the  sampling  rates  and  results 
are  discussed  in  sections  6.5.2  through  6.5.4. 

At  the  completion  of  quality  '-ontrol  processing,  the  questionnaires  were 
packed  into  boxes  and  shipped  to  the  ETJ  data  retention  arei  for  long-term 
storage. 
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Chapter  6.2 
PROFESSIOIIAL  SCORING^ 

Lynn  B.  Jenkins  and  Anne  Campbell 
Educational  Testing  Service 


Like  previous  NAEP  assessments,  the  1988  assessment  included  a  variety 
of  open-ended  items — or  items  that  ask  students  to  provide  written  responses. 
Open-ended  items  were  administered  as  part  of  the  main  assessments  in  reading, 
writing,  document  literacy,  civics,  and  U.S.  history,  and  the  bridge 
assessments  in  reading,  writing,  civics,  mathematics,  and  science.    Some  of 
the  items  requested  extended  writing,  and  these  appeared  alone  in  a  block  ?o 
that  students  had  10  to  15  minutes  to  respond.    Others  requested  shorter 
written  responses;  these  were  interspersed  with  other  items  in  a  block. 

The  1988  uiain  and  bridge  (or  trend)  assessments  included  the  foliowinjp; 
numbers  of  open-ended  items. 


1988  Main  Assessment 


Grade  4/A^e  9         Grade  8 /Age  13       Grade  12 /Age  17 

Reading  2  12 

Writing  7  8  8 

Document  Literacy  N/A  27  27 

Civics  Oil 

U.S.  History  Oil 


1988  Bridge  Crend)  Assessments 


Age  Class  9  Age  Class  13  Age  Class  17 

Reading  5  8  12 

Writing  6  6  6 

Civics  0  12 

Science  0  0  2 

Mathematics  28  27  54 


^The  authors  would  like  to  acknowledge  Debra  Kline,  Walter  MacDonald,  and 
Ina  Mullis  for  their  contributions  to  the  text  of  this  chapter  and  Bruce 
Kajilan,  David  Freund,  Reb'^cca  Zwick,  and  Jim  Ferris  for  providing  statistical 
data. 
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In  both  the  bricge  and  izcin  assessments,  some  of  the  same  items  were 
administered  at  more  than  one  age  class.    Tables  6.2-1  and  6.2-2  give  an 
overview  of  the  main  and  bridge  <xssessment  items,  including  their  NAEP 
identification  numbers,  response  time  provided,  age  classes,  and  score  ranges. 

In  the  1988  assessment,  three  types  of  answer  documents  were  used:  key- 
entered  booklets,  scannable  booklets,  and  scannable  answer  sheets.  The 
placement  of  the  scores  and  the  manner  in  which  the  scores  were  recorded 
varied  according  to  the  type  of  answer  document  that  was  used.     Scores  for 
open-ended  items  in  the  bridge  to  1984  were  recorded  on  the  back  covers  of  the 
booklets  and  subsequently  key-entered.     Machine -scannable  booklets  were  used 
for  the  bridge  to  1986,  for  the  main  assessment  of  grade  4/age  9  students  in 
all  subject  areas,  and  for  the  main  assessment  of  grade  8/age  13  and  grade 
12/age  17  students  in  writing  and  document  literacy.     Scores  for  open-ended 
items  in  these  samples  were  gridded  in  ovals  at  the  bottom  of  the  pages  on 
which  the  items  appeared.     Scannable  answer  sheets  were  used  for  the  main 
reading,  civics,  and  U.S.  history  assessments  of  students  in  grade  8/age  13 
and  grade  12/age  17.     Scores  for  open-ended  items  in  these  samples  were 
gridded  on  the  page  of  the  answer  sheet  where  the  response  was  written. 

Three  teams  of  readers  worked  simultaneously  to  score  the  open-ended 
items.    One  team  scored  responses  to  the  mathematics  bridge  items,  while  a 
second  team  scored  responses  to  the  bridge  items  in  the  other  subject  areas. 
A  third  team  scored  all  open-ended  item  responses  from  the  main  assessment. 

The  rest  of  this  chapter  includes  a  description  of  the  scoring 
operation,   including  scoring  guides,  training,  work  flow,  and  the  measures 
used  to  monitor  the  reliability  of  the  scoring  procedures. 

6.2.1  DESCRIPTION  OF  SCORING 

Each  open-ended  itew  included  in  the  1988  assessment  had  a  unique 
scoring  guide  that  identified  the  range  of  possible  scores  for  the  item  and 
defined  the  criteria  to  be  used  in  evaluating  students'  responses.    To  enable 
NAEP  to  accurately  measure  changes  in  performance  across  time,  the  scoring 
guides  for  open-ended  trend  items  readministered  in  1988  were  identical  to 
those  used  in  the  previous  assessments.    The  following  sections  summarize  the 
scoring  guidelines  used  to  evaluate  responses  to  the  open-ended  items  for  each 
subject  area  assessed  in  1988. 

Reading  and  Document  Literacy 

The  scoring  guides  for  the  open-ended  reading  items  incl  *ded  in  the 
bridge  and  the  main  assessments  focused  on  students'  ability  to  perform 
various  reading  tasks — for  example,  identifying  the  author's  message  or  mood 
and  substantiating  their  interpretation,  making  predictions  based  on  g^iven 
details,  supporting  an  interpretation,  and  comparing  and  contrasting 
information. 
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Table  6.2-1 
NAEP  1988  Main  Sample  Open-ended  Items* 


Response         Age  Class       Score  Secondary 


NAEP  ID 

DescriDtion 

Time 

(Mins.  ) 

9 

13  i; 

Ranpe 

Trait 

Writine 

N000311 

Recreation  0pp. 

15 

X 

X 

0-4, 

7-9 

Yes 

N000331 

Recreation  0pp. 

30 

X 

X 

0-4, 

7-9 

Yes 

N000411 

Food  on  Frontier 

15 

X 

X 

0-4, 

7-9 

N000551 

Dissecting  Frogs 

7*1 

X 

0-4, 

7-9 

N000941 

Radio  Station 

1  A 

10 

X 

0-4, 

7-9 

N000951 

Radio  Station 

7*1 

X 

0-4, 

7-9 

N007711 

Ghost  Story 

15 

X 

X 

0-4, 

7-9 

Yes 

N007721 

Ghost  Story 

on 

V 

0-4, 

7-9 

I  es 

N007731 

Ghost  Story 

OA 

30 

X 

X 

0-4, 

7-9 

Yes 

N007741 

Ghost  Story 

1  A 

10 

X 

0-4, 

7-9 

les 

N014741 

Plants 

1  A 

10 

X 

0-3, 

7-9 

N014821 

Spaceship 

OA 

20 

X 

0-4, 

7-9 

Yes 

N014841 

Spaceship 

10 

X 

0-4, 

7-9 

Yes 

N018051 

Space  Program 

74 

X 

0-4, 

"7  A 

7-9 

N021051 

Bike  Lane 

X 

0-4, 

7-9 

W000141 

Summary  of  Story 

10 

X 

0-4, 

9 

W000221 

Favorite  Animal 

on 
ZU 

V 

0-4, 

9 

X  es 

W000241 

Favorite  Animal 

1  A 

10 

X 

0-4, 

9 

Yes 

W000341 

Three  Wishes 

1  A 

10 

V 

A 

0-4, 

9 

W000411 

Favorite  Story 

15 

X 

X 

U-4 , 

W000511 

TV  Habits 

15 

X 

X 

0-4, 

8,  9 

Yes 

W000531 

T\'  Habits 

30 

X 

X 

0-4, 

8,  9 

Yes 

WOOOoll 

Memorable  Event 

X 

X 

0-4, 

9 

Reading 

0-5, 

R000206 

Dove  and  Ant 

(I)** 

X 

9 

R000807 

Grandpa  and  Wind 

(I) 

X 

X 

0-5, 

9 

n  A  AO  /  r\c 

R002400 

Small  Fruits 

X 

0-2, 

9 

N015905 

High  Tech  Pizza 

(I) 

X 

0-4, 

7-9 

U.S.  His 

tory 

0-5, 

H024901 

Settlers 

15 

X 

9 

H025002 

Preside  Power  Part 

1 

15 

X 

0-2 

H025003 

Presid.  Power  Part 

2 

15 

X 

0-4, 

9 

Civics 

P018201 

Presid.  Resp,  Part 

1 

15 

X 

X 

0-1 

P018202 

Presid.  Resp.  Part 

2 

15 

X 

X 

0-4, 

9 

*      Not  including  open-ended  items  scored  as  right/wrong. 

**    (I)  denotes  that  the  item  appeared  in  a  10-minute  block  at  grade  4 
or  a  IS-minute  block  at  grades  8  and  12  that  contained  several  multiple-choice 
content  items. 
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Table  6.2-2 
NAEP  1988  Bridge  Sample  Open-ended  Items* 


Response      Age  Class  Score  Secondary 

NAEP  ID         Description     Time  ^Hins,)     9  13  17  Range  Trait  Holistic 


Writing 


N000302 

Recreation  0pp. 

15 

X 

X 

0-4,  7-9 

Yes  Yes 

15 

X 

X 

0-4,  7-9 

Ves 

N000502 

Dissecting  Frogs 

7h 

X 

0-4,  7-9 

N000602 

XYZ  Company 

7h 

X 

X 

0-3,  7-9 

N000902 

Radio  Station 

15 

X 

X 

0-4,  7-9 

N001002 

Appleby  House 

15 

X 

X 

X 

0-4,  7-9 

N007602 

Flashlight 

15 

X 

0-4,  7-9 

Yes 

N014702 

Plants 

15 

X 

0-3,  7-9 

N014802 

Spaceship 

7h 

X 

0-4,  7-9 

Yes 

N018002 

Space  Program 

7h 

X 

0-4,  7-9 

N019002 

Job  Application 

7h 

X 

0-4,  7-9 

Yes 

N021002 

Bike  Lane 

15 

X 

0-4,  7-9 

Reading 

Nuts 

X 

X 

X 

0-6,  7-9 

Yes 

N001904 

Charley 

(I) 

X 

X 

0-5,  7-9 

Yes 

i.ne  uoor 

\^) 

Y 

Y 

0-9 

N002804 

Bethune 

(I) 

X 

X 

X 

0-5,  7-9 

N003104 

Goods  to  Market 

(I) 

X 

X 

X 

0-4,  7-9 

N0037O4 

Web  Life 

(I) 

X 

X 

X 

0-4,  7-9 

N004303 

Javelin 

X 

X 

0-4,  '/  9 

N004605 

Jobs 

(I) 

X 

X 

0-5,  7-y 

N008905 

Mother  and  Dog 

(I) 

X 

0-6 

Yes 

N015505 

High  Tech  Pizza 

(I) 

X 

0-4,  7-9 

N021301 

Jacob 

(I) 

X 

0-4,  9 

Yes 

N021801 

Eggplant  I 

(I) 

X 

0-5,  9 

Yes 

N021805 

Eggplant  II 

(I) 

X 

0-4,  9 

Yes 

Civics 

P021001 

Democracy 

(I) 

X 

10-15,  20- 

■24, 

77,  88 

P021101 

Newspaper  Publishers 

(I) 

X 

X 

10-16,  20- 

■21, 

7,  8 

Science 

N430801 

Pendulum 

(I) 

X 

0-3,  9 

N437001 

Battery/Bulb 

(I) 

X 

0-4,  9 

*     Not  including  open-ended  items  scored  as  right/wrong. 

**  (I)  denotes  that  the  item  appeared  in  a  10 -minute  block  at  grade  4  or  a 
15-minute  block  at  grades  8  and  12  that  contained  several  multiple- choice  content 
items . 
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The  guides  for  the  raading  items  varied  somewhat,  but  typically  included 
the  distribution  of  score  points  shown  below. 

Outline  for  Scoring  of  Open-Ended  Reading  Items 

Definition 

Elaborated  reference  or  interpretation.    These  responses 
exceeded  the  requirements  of  th^  task  by  including 
illustrative  examples  or  details  and  demonstrating  a  high 
level  of  cohesiveness . 

Satisfactory  reference  or  interpretation.    These  responses 
identified  at  least  two  relevant  examples  or  reasons  to 
support  a  given  interpretation. 

Minimal  reference  or  interpretation.  These  responses 
identified  at  lep^st  one  relevant  example  or  reason  to 
support  a  given  interpretation. 

Unsatisfactory  reference  or  interpretation.  These  responses 
did  not  give  evidence  to  support  a  stated  interpretation. 

No  reference  or  interpretation.    These  responses  did  not 
provide  an  interpretation,  but  instead  digressed  or  avoided 
the  task. 

0,  7,  8,  9         These  responses  were,  respectively,  blank,  ir^decipherable , 
completely  off-task,  or  included  a  statement  to  the  effect 
that  the  student  did  not  know  how  to  do  the  task.     (In  the 
scoring  guides  for  the  main  asse:^  .aent,  scores  of  7,  ^,  and 
9  were  collapsed  into  the  score  point  of  9). 

Some  of  the  guides  for  the  main  assessment  items  included  secondary 
scores,  which  typically  involved  categorizing  the  kind  of  evidence  or  details 
the  student  used  as  support  for  an  interpretation.    The  document  literacy 
items,  most  of  which  required  short  answers,  were  scored  on  a  right-wrong 
basis. 

Writing 

There  are  widely  divergent  views  as  to  what  constitutes  good  writing. 
In  response  to  these  different  conceptions,  writing  researchers  have  devel:)ped 
a  variety  of  methods  for  evaluating  students'  writing  abilities. 

To  provide  multiple  perspectives  on  students'  writing  performance,  NAEP 
uses  three  scoring  approaches — primary  trait,  holistic  (or  general 
impression),  and  mechanics  scoring — to  evali    :e  responses  to  the  writing 
assessment  t.sks.     Selected  writing  items  in  the  bridge  assessmpnr  were  scored 
using  all  three  approaches,  while  the  remaining  items  were  scored  using  the 
primary  trait  method  only.    The  primary  trait  method  was  used  to  score  items 
in  the  main  assessment. 
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Score 
5 

4 

3 

2 
1 


As  described  in  the  sections  that  follow,  the  purposes  of  the  three 
scoring  approaches  used  by  NAEP  are  quite  different.    Primary  trait  scoring 
focuses  on  students'  ability  to  accomplish  the  core  purpose  of  a  particular 
writing  task,  holistic  scoring  focuses  on  overall  fluency,  and  mechanics 
scoring  focuses  on  students'  grammar,  punctuation,  and  spelling.  Previous 
research  has  revealed  moderate  correlations  between  the  results  from  holistic 
and  primary  trait  scoring  (ranging  from  .29  to  .60);  however,  the  two 
approaches  evidently  capture  different  aspects  of  writing  perforirance 
(Applebee,  Langer,  &  Mullis,  1989).     The  range  of  the  correlation  coef f icier*^^* 
between  the  two  sc   '      approaches  was  from  .39  to  .66  for  the  trend  results 
for  the  1984  and  1988  writing  assessments  (Applebee,  Langer,  Mullis,  & 
Jenkins,  1990). 


Primary  Trait  (or  Task  Accomplishment)  Scoring.    As  noted  above,  the 
primaiy  trait  scoring  method  focuses  on  the  writer's  effectiveness  in 
accomplishing  specific  tasks.     It  is  sensitive  to  the  writer's  understanding 
of  the  audience  as  well  as  to  the  inclusion  of  specific  features  needed  to 
accomplish  the  specific  purpose  of  a  task.    The  primary  trait  scoring  criteria 
defined  five  levels  of  task  accomplishment:  not  rated,  unsatisfactory, 
minimal,  adequate,  and  elaborated.    The  scoring  guide  for  each  item  described 
these  levels  in  detail.    A  general  explanation  of  the  score  points  is  given 
below. 

Levels  of  Writing  Task  Accomplishment 

Score  Definition 

4  Elaborated.     Students  providing  elaborated  responses  went 

beyond  the  essential,  reflecting  a  higher  level  of  coherence 
and  providing  more  detail  to  support  the  points  made. 

3  Adequate .    Students  providing  adequate  responses  included 

the  information  aad  ideas  necessary  to  accomplish  the 
underlying  task  and  were  considered  likely  to  be  effective 
in  achieving  the  desired  purpose.     (For  two  of  the  items, 
this  was  the  highest  possible  score.) 

2  Minimal .     Students  writing  at  the  minimal  level  recognized 

sonie  or  all  of  the  elements  needed  to  complete  the  task  but 
did  not  manage  these  elements  well  enough  to  assure  that  the 
purpose  of  the  task  would  be  achieved. 

1  Unsatisfactory.     Students  who  wrote  papers  judged  as 

unsatisfactory  provide<l  very  abbreviated,  circular,  or 
disjointed  responses  that  did  not  even  begin  to  address  the 
writing  task. 

0,  7,  8,  9         Not  Rated.    A  small  percentage  of  the  responses  were  blank, 
indecipherable,  or  completely  off  tasV ,  or  contained  a 
statement  to  the  effect  that  the  student  did  not  knov  how  to 
do  the  task;  these  responses  were  not  rated. 
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Some  items  also  were  scored  for  secondary  traits,  which  involved 
indicating  the  presence  or  absence  of  elements  that  were  of  special 
significance  to  a  particular  item  (e.g.,  whether  notes  were  made  before 
writing  or  whether  critical  information  was  filled  out  on  a  form) . 


hechanics  Scoring.    Mechanics  scoring  focuses  on  the  extent  to  which  che 
writer  can  control  the  conventions  of  written  English — specifically,  grammar, 
punctuation,  and  spelling.    In  additicn,  the  procedures  include  identifying 
r^intence  structures  and  computing  sentence  length  to  gauge  the  sophistication 
of  students'  syntax.    In  the  mechanics  scoring,  conducted  after  the  main 
scoring  effort  had  been  completed,  two  teejns  of  readers  joined  together  to 
score  a  subset  of  responses  to  selected  open-ended  items  from  the  writing 
bridge  assessment.    One  item  was  chosen  at  grade  4  ("Spaceship")  and  a  second 
item  was  chosen  at  grade  8  and  grade  11  ("Recreation  Opportunities").  A 
random  probability  sample  of  approximately  500  essays  was  selected  from  each 
grade  level  for  the  1984  and  the  1988  assessment.^ ,  for  a  total  of  1,000  essays 
at  each  grade  level.    The  set  of  essays  selected  from  each  grade  level  for 
each  year  included  responses  from  t\pproximately  200  studentii  who  were  Black 
and  approximately  300  students  who  were  not.    Black  students  were  oversampled 
to  ensure  that  the  comparisons  of  performance  between  Black  and  White  students 
were  reasonably  3*recise. 

Prior  to  the  scoring,  the  responses  were  duplicated  with  the  student's 
identification  number  shown  on  ♦'he  copy.    The  essays  were  then  bundled  by 
grade  by  assessment  year.    As  i    t  readers  selected  bundles  to  score,  ^-hey 
alternated  among  the  different  grade  levels  and  years. 

Rather  than  assigning  a  single  score  to  each  paper,  as  was  done  in  the 
primary  trait  scoring,  the  mechanics  scorers  marked  each  paper  with  a  series 
of  symbols,  addressing  the  elements  of  sentence  construction,  word  choice, 
spelling,  punctuation,  and  capitalization.    These  symbols,  written  In  red  ink, 
designated  each  word  c,-  punctuation  mark  in  error  and  indicated  sentence  type 
or  faulty  sentence  construction. 

To  analyze  the  mechanics  data,  criteria  were  devised  to  derive 
information  from  --coring  codes  (see  Campbell,  1987  for  a  description  of  these 
criteria).    The  analyses  included  calculations  of: 

1)  the  average  number  of  words  in  an  er«say; 

2)  the  average  number  of  sentences  in  a.i  essay; 

3)  the  average  number  of  letters  in  a  word; 

4)  the  average  number  of  errors  in  an  essay; 

5)  the  percentage  of  different  tjrpes  of  sentence  consnructioii:  and 

6)  the  rate  of  punctuation  errors  and  omissions. 


Holistic  Scoring.     In  holistic  scoring,  readers  evaluate  the  fluency  of 
each  student's  writing  compared  to  the  writing  of  other  students  at  the  same 
grade  or  age  level  who  responded  to  the  same  task.    Unlike  primary  trait  or 
mechanics  scoring,  where  the  reader  focuses  on  the  presence  or  absence  of 
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particular  elements,  holistic  scoring  takes  a  global  view  of  the  ideas, 
l^nguage  facility,  organization,  mechanics,  and  syntax  of  each  paper  taken  as 
a  whole — as  its  name  implies.     "The  chief  assumption  that  underlies  holistic 
scoring  of  essays  is  that  the  whole  text  or  composition  is  more  than  the  sum 
of  its  parts... To  look  at  a  composition  as  a  whole  in  order  to  judge  its 
quality  as  an  entity  in  it/self  is  to  score  it  holistically"     (Ereland,  Camp, 
Jones,  Morris,  &  Rock,  1987,  p.  18). 

The  holistic  scoring  was  conducted  by  a  large  group  of  readers  in  a 
session  that  was  conducted  separately  from  the  primary  trait  and  mechau!  s 
scoring  sessions.    The  tasks  scored  holistically  were  "Spaceship"  and 
"Flasulight"  at  grade  4,  and  "Recreation  Opportunities"  and  "Food  on  the 
Frontier"  at  the  upper  grade  levels  (8  and  11).    Trained  readers  evaluated  the 
relative  fluency  of  students'  writing  on  a  6 -point  scale.    A  small  percentage 
of  papers — such  as  those  that  were  blank  or  indecipherable — were  not  rated. 

The  holistic  scale  was  anchored  by  chief  readers  and  table  leaders 
chosen  for  their  expertise  in  holistic  scoring.    This  group  studied  the  pool 
of  student  responses  to  select  papers  that  represented  each  point  on  the 
holistic  scale,  then  used  these  sample  papers  to  train  a  group  of 
approximately  50  raters.    Using  the  sample  p^*pers  as  a  guide,  the  raters  wer 
trained  to  determine  whether  papers  corresponded  to  the  top  half  or  the  bottom 
half  of  the  holistic  scale,  then  to  make  finer  distinctions  between  adjacent 
points  on  the  scale. 

To  conduct  the  scoring,  the  readers  were  divided  into  two  group  One 
large  group  was  responsible  for  evaluating  eighth  and  eleventh  grader^*' 
responses  to  the  two  tasks  common  to  those  grade  levels,  while  a  smaller 
second  group  was  responsible  fo:  evaluating  fourth  graders*  responses  to  he 
two  tasks  given  only  at  that  gtade  level.     Because  the  emphasis  of  the 
holistic  scoring  was  on  detecting  trends  across  time  at  each  of  the  tht^. 
grade  levels  assessed,  the  tasks  given  at  grades  8  and  11  were  rated 
separately,  although  by  the  same  readers.    A  training  session  preceded  the 
scoring  of  responses  to  each  task  at  each  grade  level.  ^ 

Student  papers  are  evaluated  relative  to  one  another  in  holistic 
scoring,  rat>sr  than  against  specific  criteria,  as  with  primary  trait  scoring. 
Therefore,  for  each  task  at  each  grade  level,  the  distribution  of  scores  for 
the  total  sample  of  papers  should  be  approximately  normal,  with  scores  /enly 
distributed  around  the  center  of  the  scale.    To  detect  changes  in  writing 
fluency  across  time  at  each  grade  level,  papers  from  the  1984  and  1988 
assessments  were  randomly  mixed  prior  to  scoring.    Thus,  if  more  papers  from 
either  assessment  were  judged  to  be  in  the  top  half  of  the  scale,  the  results 
would  indicate  changes  across  time  in  overall  writing  fluency. 

Mathematics 

Because  the  open-ended  mathematics  items  in  the  1988  assessment  tenJ  d 
to  focus  on  computational  skills,  all  were  scored  on  a  right -wrong  basis, 
where  1-correct  and  2-inccrrect.    Omitted  responses  were  scored  as  0.  Answers 
written  on  the  answer  lines  were  the  primary  basis  for  the  scores;  however,  if 


134 


J49 


the  stiident  left  the  answer  line  blank,  consideration  was  given  to  answers 
written  under  the  item  or  answers  written  where  che  student  had  worked  out  the 
Item. 


Science 

The  scoring  guides  jfor  the  two  open-ended  science  items  ("Pendulur.."  and 
''Batteries  and  Bulbs")  focusei  on  how  correctly  the  student  answered  the 
questions.    The  following  outline  summarizes  the  guideli.nes  used  to  Sv.ore 
these  items. 

Outline  for  Scoring       Open- Ended  Science  Items 
Definition 

This  score  indicated  a  corract,  detailed  answer.  (Only  one 
of  the  scoring  guides  included  this  score  point.) 

This  score  vrdicated  a  correct  answer. 

This  score  indicated  an  answer  that  was  correct  to  a  point 
but  either  contained  some  misinformation  or  was  too  general. 

This  score  indf  .ated  an  incorrect  response  t">  the  question. 

These  scores  were  given  to  response.s  that  were, 
respectively,  blank,  indecipherable,  or  off-task  (not 
relevant),  or  contained  a  statement  to  the  effect  that  the 
student  did  not  know  how  to  do  the  task. 


Civics  and  U.S.  History 

The  first  part  of  the  scoring  standard  for  the  open-ended  civics  item 
("Presidential  Responsibilities")  included  in  the  main  assessment  asked 
readers  to  distinguish  oetween  correct  and  incorrect  responses  to  the  initial 
part  of  ths  task,  in  which  students  were  asked  to  i  ame  the  current  president. 
The  second  part  of  the  task  asked  students  to  describe  the  prerident's 
responsibilities,  and  the  accompanying  guide  defined  the      'teria  for  each 
score  point,  as  shown  bslow. 

In  contrast,  the  scoring  rubrics  for  the  two  civics  trend  items 
("Newspaper"  and  "Democracy")  defined  specific  criteria  for  acceptable  and 
unacceptable  responses.    Many  types  of  acceptable  responses  were  possible,  and 
each  type  was  given  a  separate  score. 


Score 
4 

3 
2 

1 

0,  7,  8,  9 
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Outline  for  Scoring  of  Open- Ended  Civics  Item 


Score  Definition 

4  Elaborated.    These  responses  provide  a  mix  of  specific 
examples  and  thoughtful  discussion. 

3  Adequate.    These  responses  provide  one  or  two  examples  of 
responsibilities  with  little  discussion. 

2  Minimal.    These  responses  consist  primarily  of  generalities 
or  contain  a  list  of  information  that  contains  errors. 

1  Unacceptable.    These  responses  digress  from  the  topic,  give 
incorrect  information,  or  do  not  attempt  to  respond  to  the 
question. 

9  Not  Rated.    No  response  or  totally  off- task. 

As  shown  below,  the  scoring  guides  for  the  two  open-ended  U.S.  history 
items  included  in  the  main  assessment  ("Settlers"  and  "Presidential  Powers") 
focused  on  the  accuracy  and  elaboration  of  students'  responses  to  the 
questions.    As  with  the  civics  item  ("Presidential  Responsibilities") 
previously  described,  the  first  part  of  the  "Presidential  Powers"  task  jus 
scored  dichotomously.     Students  were  asked  to  state  who  was  more  powerful — the 
presidents  of  today  or  of  Washington's  era,  and  raters  marked  whether  or  not 
the  student  took  a  position.     In  the  second  p-art  of  the  task,  students  were 
asked  to  support  the  position  they  stated. 

Outline  for  Scoring  of  Open- Ended  U.S.  History  Items 
Score  Description 

5  These  responses  contain  several  reasons  supported  by 
appropriate,  specific  examples.     (Only  the  scoring  guide  for 
the  "Settlers"  item  specified  this  score  point.) 

4  These  responses  contain  at  le^st  two  reasons  with 
explanations  and  may  also  give  a  lengthy  li.it  with  an 
explanation  of  at  least  one  ite»9. 

3  These  responses  give  a  list  of  reasons  without  any 
explanation  or  one  reason  with  an  explanation.    They  contain 
no  significant  errors. 

2  These  responses  provide  only  one  correct  reason,  repeat  a 
single  point,  or  include  incorrect  or  insignificant  reasons. 

1  These  responses  do  not  answer  the  question  correctly  or 

reiterate  the  question. 

9  No  response  or  totally  off- task. 
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6.2.2  THE  SCORi::^  OPERATION 


Overview  of  the  Scoring  Operation 

For  the  main  assessment,  a  group  of  eight  persons  scored  the  open-ended 
items  for  all  subject  areas.     For  the  bridge  assessment,  three  persons  scored 
all  the  open-ended  items  in  all  subject  areas.    A  majority  of  the  readers  had 
at  least  bachelor's  degrees  in  education,  English,  or  history,     The  two 
persons  assigned  to  the  scoring  of  the  mathematics  items  at  all  three  levels 
had  at  least  a  high-school  education.    The  readers  included  men  and  women  of 
various  ages  and  racial,  ethnic,  and  geographical  backgrounds. 

The  NAEP  scoring  supervisor         ctly  monitored  the  scoring  of  the 
mathematics  items  and  managed  the  c^^eration  of  the  other  two  groups.  The 
scoring  supervisor  also  reviewed  discrepancies  between  readers  in  the  scoring 
of  responses  to  the  bridge  items.    To  facilitate  the  scoring  procers,  the 
supervisor  delegated  the  responsibility  of  re  iewing  scoring  discrepancies 
between  readers  for  the  main  assessment  items  to  two  of  the  best  scorers  in 
the  group.    However,  the  scoring  supervisor  was  always  available  to  consult 
with  these  individuals  when  they  encountered  responses  that  were  particularly 
difficult  to  score. 


Training:  Mathematics 

Because  the  mathematics  items  were  scored  as  right,  wrong,  or  omitted, 
lengthy  training  for  scoring  these  items  was  unnecessary.    In  an  orientation 
period,  the  readers  were  trained  to  follow  the  procedures  for  scoring  the 
mathematics  items  and  became  familiar  with  the  scoring  guides,  which  lasted 
the  correct  answers  for  the  items  in  each  of  the  blocks. 


Training:      Reading,  Writing  (Primary  Trait),  Civics,  U.S.  History, 
and  Science  Scoring 

Before  the  training  program  started,  the  NAEP  scoring  supervisor  worked 
with  NAEP  test  development  staff  to  prepare  training  sets  (or  sets  of  sample 
responses  to  accompany  the  scoring  guides)  and  to  refine  the  scoring  guides 
for  newly  developed  items. 

For  the  main  assessment,  readers  were  trained  on  all  the  writing, 
reading,  civics,  and  U.S  history  items  at  all  three  grade/age  levels. 
Training  involved  explaining  the  item  and  its  scoring  guide  and  discussing 
responses  that  were  representative  of  the  various  score  points  in  the  guide 
When  this  process  was  complete  the  readers  scored  and  then  discussed 
approximately  65  to  100  randomly  selected  "practice  papers"  for  each  item. 
The  purpose  of  the  training  was  to  familiarize  the  group  with  the  scoring 
guides  and  to  reach  a  high  level  of  agreement  among  '-he  readers.    When  the 
group  craining  had  been  completed,  each  reader  scored  all  the  open-ended  items 
in  each  of  nine  bundles  of  booklets,  after  which  a  follow-up  session  was  hel-. 
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to  discuss  responses  that  received  a  wide  range  of  scores.  Once  the  follow  up 
session  was  completed,  the  formal  scoring  process  began.  The  initial  training 
was  completed  in  approximately  four  weeks. 

The  training  program  for  the  bridge  assessment  was  carried  out  on  all 
the  items  st  one  age  class  at  a  time,  starting  with  age  class  13,  followed  by 
age  class  9,  and  ending  with  age  class  17.     (This  order  corresponds,  to  the 
order  in  which  the  bridge  assessments  are  conducted.)    The  training  program 
followed  the  same  procedures  used  for  the  main  assessment  scorin£,.     In  order 
to  ensure  continuity  with  the  past  scoring  uf  the  bridge  items,  at  least  half 
the  sample  papers  in  the  training  sets  were  taken  from  the  1984  training  sets. 
The  training  program  for  each  age  level  took  approximately  two  weeks. 

As  a  follow-up  to  the  training  program,  notes  on  various  items  were 
compiled  for  the  readers  of  each  group  for  their  reference  and  guidance 
throughout  the  scoring  process.     In  addition,  short  training  sessions  were 
conducted  when  the  scoring  supervisor  ascertained  in  reviewing  discrepancies 
that  certain  items  were  causing  difficulties  for  the  scorers.    The  scoring 
supervisor  also  'consulted  with  individual  readers  as  the  scoring  progressed. 
When  a  reader's  score  vras  judged  to  be  disciepant  with  that  of  another  reader, 
the  supervisor  discussed  the  response  and  its  score  with  that  reader. 

Training:    Writing  Mechanics  Scoring 

To  prepare  for  the  mechanics  training,  the  NAEP  scoring  supervisor 
selected  papers  to  be  used  in  training.    The  training  itself  involved 
discussing  the  scoring  guidelines  and  procedurc-s  and  reviewing  sample 
responses  that  had  already  been  scored.     The  readers  then  practiced  scoring 
other  papers,  and  discussions  were  held  when  any  discrepancies  occurred.     '  .-en 
the  readers  were  comfortable  with  the  guidelines,  the  actual  scoring  began. 
Sevoral  follow-up  training  sessions  were  held  as  problems  arose. 

Training:    Holistic  Scoring 

The  training  for  the  holistic  scoring  of  writing  bridge  items  involved 
several  stages.     First,  NAEP  staff  developed  guidelines  describing  six  levels 
of  proficiency  for  each  task.    Then,  NAEP  staff  and  two  chief  readers — both  of 
whom  were  expeii-enced  holistic  readers — surveyed  the  pool  of  pc.pers  from  the 
assessments  and  selected  anchor  papers,  or  papers  representative  i     the  six 
levels  of  proficiency.    The  guidelines  were  modified  accordingly  and  criteria 
were  established  for  distinguishing  between  top-half  and  bottom-half  papers. 
A  session  was  then  held  for  the  table  leaders — who  v/ere  also  experienced 
holistic  readers — to  familiarize  them  with  the  guidelines  and  sample  papers. 

The  training  of  the  readers  began  with  some  discussion  of  the  guidelines 
and  the  anchor  papers  and  included  several  practice  scorings  of  other  p^^pers 
to  resolve  discrepancies  among  uaders.    When  all  the  readers  u^^re  comfortable 
with  the  guidelines,  they  scored  papers  fo^  an  hour,  after  whici.  they 
discussed  additional  anchor  papers.    Throughout  the  subsequent  scoring,  ^here 
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were  periodic  discussions  of  papers  t  ensure  that  readers  continued  to  adhere 
to  the  same  standards. 


Assignment  of  Work 

The  two  groups  of  readers  for  the  bridge  assessment  began  scoring  the 
age  class  13  items  in  November  1987.    These  same  two  groups  started  scoring 
the  age  class  9  items  in  February  1988,  and  the  age  class  17  items  in  April. 
The  readers  for  the  main  asset>sment  started  scoring  in  March.     Each  group  of 
readers  received  the  booklets  in  batches  as  they  were  received  from  the 
schools.    Because  of  the  spiral  design,  a  reader  would  receive  many,  if  not 
all,  of  the  items  at  a  particular  age  class  as  he  or  she  scored  a  batch  of 
booklets.     In  scoring  the  main  assessment  items,  the  scorers  alternated 
through  the  thre^  age  classes  so  that  they  were  continually  exposed  to 
responses  from  .21  age  classes  throughout  the  scoring. 

6*2.3    RELIABILITY  AND  RESOLUTION 
Trend  Assessments 

Twenty  percent  of  the  1984  and  1986  responses  to  the  open-ended  reading 
items  in  the  bridge  assessments  were  retrieved,  the  scores  were  masked,  sad 
the  responses  were  disurijuted  to  and  rescored  by  the  reader.s.     This  lescoring 
was  performed  concurrently  with  the  scoring  of  the  1988  responses.     Because  of 
differences  in  the  way  that  scoring  guidelines  were  applied  to  open-ended 
reading  and  writing  responses  in  1988  and  previous  assessments,  the  1988 
results  for  the  professionally  scored  items  are  not  directly  comparable  to 
past  results.     Changes  in  percents  correct  for  these  items  are  not  appropriate 
for  inferences  about  changes  in  reading  or  wilting  achievement.     (See  Chapters 
10  and  11  for  further  discussion. ) 


Main  Assessments:    Reading,  Writing,  U.S.  History,  Civics,  and  Science 

Twenty  percent  of  the  items  in  the  other  subject  areas  were  subjected  to 
a  reliability  check,  which  entailed  a  scoring  by  a  second  reader.    To  prevent 
a  second  reader  from  being  influenced  by  the  first  reader's  scores,  the  first 
reader  masked  the  scores  in  every  fifth  booklet  in  a  batch.    These  booklecs 
were  passed  along  to  a  second  reader.    All  discrepancies  were  then  reviewed  by 
the  scoring  supervisor  or  those  designated  by  the  scoring  supervisor 

Mathematics 

Ten  percent  of  the  mathematics  items  were  subjected  to  a  correctness 
check  in  which  a  second  scorer  verified  that  the  first  scorer  had  correctly 
scored  the  items.     If  the  second  scorer  found  a  mistake  in  scoring,  he  or  she 
corrected  it.     To  assess  the  reliability  of  each  scorer,  the  second  scorer 
kept  count  of  the  number  of  times  he  or  she  checked  each  of  the  other  scorers 
and  the  number  of  times  he  or  she  had  to  correct  a  score.    This  procedure  was 
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followed  because  the  mathematics  items  were  scored  as  right,  wrong,  or  omitted 
and  because  the  scoring  guides  were  exact  as  to  the  correct  answers.  Results 
of  this  correctness  check  showed  that  the  first  scorer  was  correct  99  percent 
of  the  time . 

Two  statistics  were  used  to  report  reader  reliability:     the  percent  of 
exact  agreement  and  the  reliability  coefficient.     The  percent  of  exact 
agreement  is  the  percentage  of  times  that  the  two  xczd^vs  agreed  exactly  in 
their  ratings.    The  reliability  coefficir  t  is  the  intraclass  relation 
between  readers. ^    The  results  for  each  a^<:  class  are  shown  in  .co' as  6.2-3 
and  6.2-4.    The  first  column  lists  the  t^- -^ber  of  responses  analyzed;  the 
second  columi.  lists  the  percentage  of  eAc*ct  agreement  between  the  first  and 
second  readers;  and  the  third  column  is  the  reliability  coefficient. 

The  reliability  results  generally  show  a  high  level  of  agreement  between 
readers.    The  percentage  of  exact  agreement  among  readers  was  at  or  above  70.7 
percent  for  all  but  the  trend  holistic  scoring,  and  several  items  showed 
agreement  as  high  as  99  percent.    The  reliability  coefficients  were  also  high, 
ranging  from  .64  to  .99. 

The  percentage  of  exact  agreement  between  the  first  and  second  readers 
tended  to  be  slightly  lower  in  the  holistic  scoring  than  in  the  other  types  of 
scOi.ing.    However,  the  reliability  coefficients  (ranging  from  .65  to  .83)  did 
not  differ  substantially  from  those  for  the  primary  trait  scoring  and  are 
generally  as  high  or  higher  than  those  repo'-ted  for  other  studies  (Breland  et 
al.,  1987).    Also,  when  agreement  between  adjacent  score  points  was  taken  into 
consideration — that  is,  when  readers  did  not  differ  by  more  than  one  score 
point  on  the  6-point  scale — the  percent  of  agreement  for  holistic  scores 
ranged  from  88  to  94  percent. 
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^The  intraclass  correlation  coefficient  (ICC)  is  calculated  as 

MSS  -  MSR 


r(ICC)  - 


MSS  -f  MSR(K-l) 


where  MSS  and  MSR  are  the  mean  squares  for  subjects  (ratees)  and  the  mean 
square  residual  obtained  from  a  one-way  ANOVA,  and  K  is  the  number  of  raters. 
(In  the  present  application,  K-1  -  1.)    This  provides  a  consistent  (but 
biased^  estimate  of 


p(ICC)  -   ^ 


and  is  therefore  interpretable  as  the  proportion  of  total  variance  due  to 
differences  among  subjects.    The  »*rror  term  contains  the  rater  effectsT  which 
cannot  be  estimated  separately  because  subjects  were  not  rated  by  the  same  set 
of  ratf  rs. 
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Table  6.2-3 


Percentages  of  Exact  Score  Point  Agreement  and  Intraclass  Correlation  Coefficients 

for  Open-ended  Itens  in  tho  1988  Main  Samples 


NAEP  ID 
Writing 


tyescriptton 


Grade  4/Age  9 
li    X  Apree  Rel. 


Grade  8/Age  13 
fi    X  Agree  Rel. 


Grade  12/Age  17 
H    jLAgree  Rel. 


Rec.  0pp.  (15  Min. ) 

474 

9i 

.8 

0 

.93 

Rec.  0pp.   (30  Min. ) 

190 

96 

.8 

0 

.98 

MA  Art /.I  1 

Food  on  Frontier  (15  Min.) 

484 

87 

.6 

0 

.88 

MA AAC  C  1 

Dissecting  Frog  5  (7h  Min.) 

636 

91 

.8 

0 

.91 

MA A AO A  1 

Radio  Station  (10  Min.) 

709 

94 

.9 

0 

.96 

MA A AO  C 1 

Radio  Station  {Ih  Min.) 

552 

93 

.3 

0 

.94 

MAA771 1 

Ghost  Story  (15  Min.) 

450 

92 

.2 

0 

.91 

WUU/ /Zi 

Ghost  Story  (20  Min.) 

181 

89 

.0 

0 

.85 

WUU/ / ji 

Gnost  Story  (30  Min.) 

151 

95 

4 

0 

.95 

N007741 

vyiiuat.  ov.ui.jf    ^XV/   nj.li.  ^ 

Ann 

yu 

f\ 
yj 

0 

OA 

N014741 

Plants  (10  Min.) 

433 

93 

1 

0 

95 

N014821 

Spaceship  (20  Min.) 

214 

91 

6 

0 

95 

N01<>841 

Space^,hip  (10  Min.) 

433 

92 

2 

0 

95 

N018051 

Space  Program  {Ih  Min.) 

N021051 

Bike  Lane  (74  Min.) 

W000141 

Summary  of  Story  (10  Min.) 

649 

88. 

9 

0. 

87 

W000221 

Favorite  Animal  (20  Min.) 

213 

91. 

6 

0. 

95 

W000241 

Favorite  Animal  (10  Min.) 

420 

89. 

8 

0. 

91 

W000341 

Three  Wishes  (10  Min.) 

618 

92. 

4 

0. 

92 

W000411 

Favorite  Story  (15  Min.) 

669 

93. 

0 

0. 

89 

W000511 

TV  Habits  (15  Min.) 

445 

93. 

3 

0. 

95 

W000531 

TV  Habits  (30  Min.) 

225 

7 

0. 

93 

tf 000611 

Memorable  Event  (15  Min.) 

608 

87. 

3 

0. 

89 

414 

89 

.1 

0 

.92 

154 

92 

.2 

0 

.95 

415 

88 

.0 

0 

.88 

370 

92 

.2 

0 

.91 

164 

90 

.9 

0 

.92 

477 

91 

4 

0 

94 

576 

83 

9 

0. 

89 

584 

90. 

1 

0. 

91 

401 

90. 

3 

0. 

92 

203 

95. 

1 

0. 

97 

565 

88. 

1 

0. 

91 
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Table  6.2-3  ( -.ontinued) 


NAEP  ID 


Percentages  of  Exact  Score  Point  Agreement  and  Intraclass  Correlation  Coefficients 
for  Open-ended  Items  in  the  1988  Main  Samples 


Description 


RO002O6  Dove  and  Ant 

RO008O7  Grandpa  and  Wind 

RO024O6  Small  Fruits 

N0159O5  High  Tech  Pizza 


Grade  4/Age  9 
Ji    X  Agree    Rel . 


433 
437 


88.9 
93.4 


0.95 
0.92 


Grade  8 /Age  13 
K    %  Agree  Rel. 


470      97.5  0.95 


Grade  12/Age  17 
%  Agree  Rel. 


516 
409 


97.7 
91.9 


0.96 
0.93 


U.S.  History 


H0249O1      Settlers  to  America 
HO250O2      Presidential  Power  Part  1 
HO250O3      Presidential  Power  Part  2 


530      90.2  0.93 


423 
403 


92.7 
87.6 


0.78 
0.92 


Civics 


P0182O1  Presid.  Resp.  Part  1 
P018202     Presid.  Resp.  Part  2 


623 
584 


99.5 
88.9 


0.97 
0.85 


574 
556 


99.7 
90.8 


0.97 
0.91 


158 


159 
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Table  6.2-4 


Percentages  of  Exact  Score  Point  Agreement  and  Intraclass  Correlation  Coefficients 
for  Open-ended  Items  in  the  1988  Trend  Samples 


/lAEP  ID 

Grade  4/Age  9 

Grade  8/Age  13 

Grade  11/Age  1> 

n^<2 PT""f  Tit"!  nn 

a. 

%  Agree 

Rel.. 

M 

X  Agree 

Kei . 

X  Aeree 

Rel. 

■  1  ~ 

N00O302 

Q  0  R 

U .  oZ 

Z93 

90 

.8 

A  AO 

0 . 93 

N00O371 

o  A 

290 

56.2* 

0.  76 

239 

47 

.3* 

0.66 

roou  on  r roncier 

79 . 9 

0 . 69 

Z60 

93 

.1 

0.  86 

N00O471 

ruuu  uii  rrcnuj-er  ^noxj.ouj.c^ 

O  A  O 

AO     A  U. 

0 .71 

253 

49 

.4* 

0.  65 

N00O502 

/D  .  1 

U .  0^ 

N00O602 

W  7  Co  mn  An v 

Z/ J 

n  QQ 

93.5 

0. 9Z 

N00OQ02 

ixclvlXU  ol^aUJ-Uil 

Ql  A 

ft7  n 

u .  oy 

N001002 

AoDlebv  Hoiise 

227 

90.3 

0.92 

288 

75.4 

0.69 

253 

89 

3 

0.89 

N007602 

Flashlight 

136 

87.5 

0.88 

N007608 

Flashlight  (holistic) 

163 

54.0* 

0.83 

K014702 

Plants 

350 

94.3 

0.95 

N014802 

Spaceship 

306 

91.8 

O.S"- 

N014808 

Spaceship  (holistic) 

236 

52.1* 

0.  59 

N018002 

Space  Program 

296 

89 

9 

0.93 

N019002 

Job  Application 

286 

92.3 

0.92 

N021002 

Bike  Lane 

298 

84. 

9 

0.87 

*  Note:     Primary  trait  scoring  was  on  a  4-point  scale;  holistic  scoring  was  on  a  6-polnt  scale.  For 
the  holistic  scoring,  percentages  of  agreement  between  adjacent  score  points — that  Is,  when  readers  did  not 
differ  by  more  than  one  score  point  on  the  6-polnt  scale — were  as  follows: 


Age  9 


Age  13 


Age  17 
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N00O371  Recreation  0pp.  (holistic) 

N00O471  Food  on  Frontier  (holistic) 

N007608  Flashlight  (holistic) 

N014808  Spaceship  (holistic) 


bpac 

J6 


{) 


93.9% 
94.1% 


143 


93.8% 
90.3% 


94.1% 
88.1% 
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Table  6.2-4  (continued) 


Percentages  of  Exact  Score  Point  Agreement  and  InUraclass  Correlation  Coefficients 
for  Open-ended  Items  in  the  1988  Trend  Samples 


Descriotion 

Readine 

N001507 

Nuts 

N001904 

Charley 

N002804 

Bethune 

N003104 

41  V  V  «y  JL  V 

N003704 

Web  Life 

nv/UH  J  u  J 

J  a V e  X.  i.n 

N004605 

Jobs 

N008905 

Mother  and  Dog 

N015905 

N021301 

Jacob 

MOOT  ftm 

cggp  xauu  X 

NO21802 

Eggplant  I  (Sec.  Trait) 

N0218O5 

Eggplant  II 

Civics 

P021001 

Democracy 

P021101 

Newspaper  Publishers 

Science 

N430801 

Pendulum 

N437001 

Battery /Bulb 

Grnde  4/Age  9 


Grade  8/Age  13 


%  AgJT^e 

m. 

X  Aeree 

Eel. 

N 

X  Aeree 

pel, 

144 

90.3 

0.94 

174 

82 

n 
,  A 

0 

.82 

127 

92 

.1 

0.92 

133 

70 

.7 

0 

.86 

137 

93 

.4 

0.97 

163 

92.6 

0.94 

169 

76. 

.3 

0 

.69 

146 

93 

.8 

0.92 

378 

95.0 

0.94 

348 

8]  . 

.9 

0 

.7) 

2'-7 

90 

.7 

0.89 

Hi 

94.  7 

JlL.O 

74. 

.6 

0. 

.  68 

ion 

90 

.8 

0. 93 

130 

82. 

.3 

0. 

.77 

99 

97 

.0 

0.97 

149 

85, 

.2 

0. 

.91 

114 

93 

.0 

0.93 

135 

89.6 

0.71 

98 

96. 

.9 

0.96 

1  OA 

95, 

.6 

0 . 98 

162 

96, 

.3 

0.98 

131 

95, 

.4 

0.95 

140 

98. 

6 

0.92 

371 

98. 

1 

0.98 

203 

90. 

1 

0. 

90 

370 

96. 

2 

0.96 

150 

97. 

3 

0.99 

155 

99. 

4 

0.99 

162 
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Chapter  6.3 
DATA  TRANSCRIPTION  SYSTEMS 


Alfred  M.  Rogers 
Educational  Testing  Service 


The  transcription  of  the  student  response  data  into  machine -readable 
form  was  achieved  through  the  use  of  three  separate  systems:  scanning, 
loading,  and  resolution. 

The  student  instruments  were  printed  in  a  format  chat  allowed  the 
transcription  of  marked  responses  in  the  booklets  to  comp*  ter  readable  form  on 
a  magnetic  tape  by  a  programmable  optical  scanning  mach'  .       The  first  part  of 
this  chapter  will  describe  the  scanning  equipment,  the  programs  and  data  used 
by  the  machinery,  and  the  ETS  quality  control  standards  and  procedures. 

A  second  procedure  "loaded"  the  data  records  from  the  scanning  output 
tape  into  an  interactive  computerized  data  entry  and  resolution  system.  This 
loading  procedure  validated  each  scanned  data  field,  reformatted  the  data 
records  to  be  compatible  with  the  resolution  system,  and  reported  all  problems 
for  subsequent  resolution.    The  second  part  of  this  chapter  details  the 
loading  procedure. 

modified  form  of  the  data  entry  system  developed  for  the  1986 
assessment  was  used  for      solution  of  the  scanned  data,  entry  of  the  docuuents 
rejected  by  the  scanning  .achine,  and  entry  of  the  questionnaire  instruments. 
The  third  part  of  thii*  chapter  will  provide  an  overview  of  this  .<5ystem,  which 
is  described  by  Rogers  (1987). 

Figure  6.3-1  is  a  schematic  diagram  that  represents  the  flow  of 
student-related  assessment  materials  through  the  data  transcription  system 
Figure  6.3-2  similarly  represents  the  flow  of  questionnaire  materials  through 
the  system.    The  reader  may  refer  to  these  diagrams  for  clarification  of  the 
relationships  among  the  components  of  this  system. 


6.3.1  SCANNING 

The  student  booklets    ^re  scanned  on  a  National  Computer  System  W201 
scanning  system.    The  scanner  was  controlled  by  a  Hewlett  Packard  1000 
minicomputer.    This  system  also  included  a  disk  drive  for  storage  of  the 
scanning  programs,  a  tape  drive  for  the  output  of  scanned  date  v^cords,  and  a 
printer  for  the  periodic  listing  of  indiviciual  record  contents  £>r  quality 
control  checking.    The  scanning  programs  uced  were  specifically  written  for 
NAEP  using  the  assembler  language  of  the  Hewlett  Packard, 
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Figure  6.3-1 


1988  NAEP  Data  Transcription  System 
(Part  1:    Student-Related  Materials) 


«niry 
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Figure  6.3-2 


1988  NAEP  Data  Transcription  System 
(Part  2:    Questionnaire  Materials) 


r" 
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An  optical  scanner  operates  by  sweeping  a  horizontally  oscillating  light 
beam  across  a  vertically  moving  sheet  and  detecting  reflections  of  the  beam 
from  pencil  marks.    The  hai    are  logic  of  the  scanner  treats  the  page  as  a 
rectangular  array  of  scannable  areas,  each  of  which  is  assigned  a  reflectance 
value  from  0  to  15.    This  array  of  values  is  passed  to  the  scanning  pr^-^ram 
software,  to  be  translated  into  response  data. 

After  the  first  side  of  a  sheet  has  been  scanned,  it  is  pushed  through  a 
loop  .:hat  brings  the  other  side  of  the  sheet  to  face  the  scanning  beam.  A 
similar  array  of  reflectance  values  is  passed  to  the  program  that  must  then 
not  only  translate  it  into  data,  but  decide  whether  to  route  this  page  to  the 
output  hopper  and  read  in  the  next  sheet  or  route  it  to  the  shunt  hopper  and 
stop  processing. 

The  paper  and  inks  used  in  producing  scannable  documents  are  required  to 
have  very  low  reflectances.    A  special  set  of  marks  are  printed  down  one  side 
of  each  page  at  equally  spaced  intervals  to  enable  the  scanning  hardware  to 
align  each  sheet  and  adjust  the  scanning  rate  to  the  movement  of  the  sheet. 
These  timing  marks  are  printed  using  a  highly  reflective  ink. 

Each  page  of  each  item  block  has  its  own  unique  format  in  terms  of  the 
arrangement  of  the  response  and  scoring  ovals.    The  scanning  program  has  to  be 
able  to  identify  a  given  page,  determine  which  parts  of  the  returned  array  to 
process,  interpret  the  reflectance  values,  and  transcribe  them  to  data  codes 
on  the  outpat  record.     Each  page  is  printed  with  a  set  of  marks  next  to  the 
timing  marks  that  are  used  by  the  program  co  identify  it  uniquely  by  block 
code  and  page  number.    The  booklet  covers  are  similarly  identified  according 
to  booklet  number. 

The  scanning  program  logic  uses  two  sets  of  tables  to  control  scanning 
processing.    When  a  booklet  cover  is  scanned,  the  program  uses  the  booklet 
number  and  the  first  table  to  drtermine  which  blocks  are  to  b&  processed. 
Each  block  code,  in  turn,  is  referred  to  the  second  table  to  determine  the 
number,  formats,  and  seque^ce  of  its  constituent  pages.    By  reading  the 
bookle'c  cover,  the  program  "knows"  which  pages  would  follow  and  in  what  order. 

The  scanning  piogram  rejects  a  page  if  it  is  unreadable  or  out  of 
sequence.    A  page  is  unreadable  if  the  timing  or  identification  marks  have 
been  corrupted  by  either  tearing,  improper  trimirlng,  or  confusing  stray  pencil 
marks.     If  the  unreadable  page  happens  to  be  a  booklet  cover,  the  operator 
instructs  the  scanner  to  send  the  remaining  pages  of  tl.at  booklet  into  the 
shunt  hopper,  places  the  p^.ges  perpendicularly  on  top  rf  the  output  stack,  and 
resumes  processing,  with  the  next  booklet.    For  any  other  page  type,  the 
operator  instructs  the  program  to  substitute  question  marks  for  the  data 
value >  on  the  unreadable  page  and  proceed  with  the  next  page. 

Pages  out  of  sequence  are  generally  attributable  to  collating  errors  in 
printing.    When  the  program  encounters  tliic  type  of  error,  the  operator 
directs  the  scanner  to  shunt  the  remaining  pages  of  the  booklet  and  places 
them  perpendicularly  on  the  output  stack. 
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The  scanning  program  writes  three  types  of  data  records  onto  the 
magnetic  tape.    The  first  is  a  batch  header  record,  containing  information 
gridded  onto  the  batch  header  sheet  by  receipt  processing  staff.    The  second 
is  a  data  record  containing  all  of  the  translated  marked  o/als  from  all  pages 
within  a  booklet      The  third  type  is  a  dummy  data  record,  serving  as  a  place 
holder  in  the  file  for  a  booklet  with  an  unreadable  cover  sheet.    The  origin 
code  is  a  data  field  writcen  in  the  same  location  on  all  records  to 
distinguish  them  by  type. 

The  batch  header  record  pr**cedes  all  data  records  for  a  given  batch.  As 
the  scanning  program  processes  the  header  sheet,  it  retains  the  batch 
identification  code  and  initializes  a  sequence  nximber  or  counter  for  that 
batch.  The  batch  identification  code  and  sequence  number  are  written  to  each 
record;  the  batch  header  record  always  receives  a  sequence  number  of  one,  the 
first  data  record  is  assigned  numbe-  two,  and  so  forth.    The  scanning  machine 
is  directed  to  stamp  the  batch  identification  code  and  sequence  number  on  each 
page  of  a  booklet.    This  process  greatly  facilitates  the  location  of 
individual  pages  within  batches  by  resolution  staff. 

Each  data  record  is  formed  by  collecting  the  transcribed  marked  ovals 
from  each  page  Oi.  a  booklet,  placing  them  into  a  buffer  area  within  the 
program,  and  writing  the  buffer  to  tape  when  the  last  page  of  the  booklet  has 
been  processed.     Several  options  were  considered  in  designing  the  format  of 
the  output  data  records.    A  format  that  requires  a  fixed  column  position  for 
each  item  response  value  would  be  very  large,  because  of  the  number  of  items 
in  the  assessment,  and  very  sparse,  because  of  the  BIB  spiral  design.  A 
format  that  has  the  response  data  strung  out  in  contiguous  fields  across  items 
and  blocks  is  more  consistent  with  the  format  of  the  data  records  in  the  NAEP 
data  entry  system,  but  would  be  difficult  to  check  in  listings  for  quality 
control.     The  format  adopted  for  this  assessment  has  fixed  column  positions 
for  the  booklet  cover  data  fields  and  scorer  identification  codes.  The 
response  data  starts  at  fixed  positions  for  each  block  within  an  instrument, 
and  the  item  responses  are  arranged  in  contiguous  fields. 

The  data  values  from  the  booklet  covers  and  scorer  identification  fields 
are  coded  as  numeric  data.    Unmarked  fields  are  coded  as  hyphens  (-)  except 
for  the  race/ethnicity,  gender,  grade,  and  birth  date  fields,  which  are 
returned  as  question  marks  (?)  to  alert  processing  staff  of  missing  or  uncoded 
critical  data.     Fields  that  have  multiple  marks  are  coded  as  asterisks  (*) . 
The  data  values  for  the  item  responses  and  scores  are  returned  as  alphabetic 
codes.    The  multiple -choice ,  single- response  format  items  are  assigned  code^ 
depending  on  the  position  of  the  response  alternative;  that  is,  the  first 
choice  is  assigned  the  code  "A",  the  second  "B",  and  so  forth.     The  circle- 
all-  that  -apply  items  are  given  as  many  data  fields  as  response  alternatives, 
the  marked  choices  are  coded  as  "A"  and  the  unmarked  choices  as  hyphens.  Tiie 
open-ended  items  have  10  ovals  labeled  from  zero  to  nine;  a  marked  zero  is 
coded  as  "A",  a  marked  one  as  "B'' ,  and  so  on  up  to  "J".    As  with  the  cover 
data  fields,  unmarked  responses  are  coded  as  hyphens  and  multiple  marks  .s 
asterisl*?.    The  fields  from  unreadable  pages  are  coded  as  question  marks  again 
as  a  fifcig  for  resolution  staff  to  correct. 
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6 • 3 . 2    DATA  LOADING  SUBSYSTEM 


Each  magnetic  tape  produced  by  the  scanning  system  contains  data  for  one 
or  more  assessment  sessions  for  one  of  the  age  groups.    The  data  records  on 
these  tapes  conform  to  a  fixed  format.    These  data  now  have  to  be  edited  for 
type  and  range  of  response ,  transformed  to  a  compressed  format  compatible  with 
the  data  entry  i>ystem  files,  and  loaded  into  the  database  for  resolution 
processing.    A  procedure  for  accomplishing  all  of  these  tasks  was  designed  and 
developed  for  this  assessment. 

Ihe  data  records  on  the  scanning  output  tape  are  ordered  in  the  same 
sequence  as  the  paper  materials  v/ere  processed  by  the  scanner.    A  record  for 
the  batch  header  precedes  all  data  records  belonging  to  that  batch;  eoch  set 
of  records  belonging  to  one  batch  are  separated  from  the  others  by  its  batch 
header  record.    The  origin  code  field  on  each  record  serves  to  distinguish  the 
header  records  from  tho  data  records. 

The  processing  of  each  batch  begins  with  the  identification  of  the 
header  record.     The  batch  identification  number  on  this  header  record  provides 
the  link  between  the  subsequent  data  records  on  tlie  tape  and  the  tracking  file 
generated  by  the  school  worksheet  entry  program  in  the  data  entry  system.  The 
load  program  uses  the  batch  identification  number  to  locate  and  retrieve  the 
processing  information  for  that  batch  from  the  tracking  file.    The  program 
then  verifies  that  it  had  the  correct  batch  by  comparing  the  PSU,  school,  and 
session  codes  gridded  on  the  header  record  with  the  s^rae  codes  in  the  tracking 
record. 

If  a  batch  code  can  not  be  located  in  the  tracking  file,  the  program 
generates  a  new  tracking  record,  using  only  the  information  contained  on  the 
'  ^ader  record,  and  records  this  condition  on  an  error  log  file.     If  a  batch 
code  is  located  but  the  school  or  session  codes  do  not  agree,  the  program 
records  this  conflict  in  the  error  log  and  continues  processing. 

The  batch  header  record  also  conta'ns  the  date  that  the  session 
materials  were  batched  together,  and  th.    .umber  of  booklets  batched  by  the 
receipt  processor.    This  information  is  transferred  to  the  tracking  record  for 
later  processing  and  reporting. 

The  reading  oi  a  br.tch  header  record  also  initiates  the  generation  of 
two  new  files  in  the  entry  system  dataoase:  the  data  file  and  the  audit  file. 
As  the  program  processes  each  recrrd  within  a  batch  from  the  tape  file,  it 
writes  the  edited  and  reformatted  data  records  to  the  data  file  and  records 
all  errors  and  special  codes  in  the  audit  file.    The  data  fields  on  an  audit 
file  record  identify  each  data  problem  by  the  batch  sequence  number,  booklet 
serial  number,  section  or  block  code,  field  name  or  item  number,  and  data 
value.     The  program  generates  a  listing  of  the  data  problems  after  each  uatch 
has  been  processed,  to  be  printed  at  the  termination  of  the  program. 

As  the  program  processes  each  data  record,  it  first  reads  the  booklet 
ntomber  and  checks  it  against  the  batch     ^ssion  code  for  appropriate  session 
type  (main  or  bridge).    Any  mismatch  is  recorded  in  the  error  log  and 
processing  continues.    The  booklet  number  is  then  compar  d  against  the  first 
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two  digits  of  '.he  student  identification  number.     If  they  disagree,  because  of 
improper  gridding,  a  message  is  written  to  the  error  log  and  the  booklet 
ntunbei  is  substituted  for  part  of  the  student  number      Tne  remaining  booklet 
cover  fields  are  then  read  and  validated  for  range.    The  PSU  and  school  codes 
must  be  identical  to  those  on  the  tracking  record;  the  range  of  grade  codes  is 
dependent  on  the  age  cohort  being  processed;  and  the  range  of  birth  dates  is 
dependent  upon  the  session  type  as  well  as  the  age  cohort.    All  data  values 
that  are  out  cT  range  are  replaced  with  question  marks  and  recorded  on  the 
audit  file.    All  data  fields  that  are  read  in  as  question  marks  or  asterisks 
are  also  recorded  in  the  audit  file.     xne  booklet  cover  data  fields  are 
written  to  a  batch  listing  file  that  is  printed  at  the  end  of  load  processing. 
This  listing  can  be  compared  against  the  administration  schedule  to  assist  in 
resolving  booklet  cover  data  problems. 

The  scorer  identification  fields  are  processed  at  this  point  and  certain 
checks  are  made.     If  a  booklet  contains  any  open-ended  items,  the  first  scorer 
field  should  be  filled.     If  a  booklet  is  part  of  the  reliability  sample,  the 
second  scorer  field  should  be  filled.     The  program  has  to  determine  from  the 
booklet  number  whether  the  booklet  contains  any  open-ended  items.  It  then 
flags  as  erroneous  any  incomplete  field  that  should  be  filled,  or  ary  nonblank 
field  that  shoulc'  be  blank  and  records  the  error  in  the  audit  file.  Further, 
it  remembers  how  many  scorer  fields  are  marked  for  later  processing  of  the 
open-ended  itf^  scores. 

The  edited  booklet  cover  and  scorer  identification  fields  are  appended 
to  the  batch  sequence  number  and  transferred  to  an  output  buffer  area  within 
the  program.    As  the  program  processes  each  block  of  data  from  the  tape 
record,  it  append?  the  edited  data  fields  to  the  data  already  in  this  buffer, 
xne  output  data  record    in  this  "compressed"  format,  is  tnus  made  compatible 
with  the  NAEP  data  entry  system. 

The  program  is  now  prepared  to  cycle  through  the  data  areas 
corresponding  to  the  item  blocks.    The  task  of  translating,  validating,  and 
reporting  errors  for  each  Jata  field  in  each  block  is  performed  by  a 
subroutine  that  requires  only  the  block  identification  code,  the  string  of 
input  data,  and  the  number  of  scorers  who  gridded  the  appropriate 
identification  fields  for  that  block.     This  routine  has  access        m  internal 
table  thac  has,  for  each  block,  the  number  of  fields  to  be  processed,  and,  for 
each  fiel''    the  field  type  (alphabetic  or  numeric),  the  field  width  in  the 
data  recc        ^  id  the  valid  range  of  values.    The  routine  then  processes  each 
field  in  sequence  order,  performing  the  necessary  translation,  validation,  and 
reporting  tasks. 

The  first  of  these  tasks  checks  for  the  presence  of  hyphens,  asterisks 
or  question  marks.    Fields  containing  asterisks  and  question  marks  are 
recorded  in  the  audit  file  and  processing  continues  with  the  next  field.  No 
action  is  taken  on  hyphen-filled  fields  inasmuch  as  that  code  indicates  a 
nonresponse.     The  field  type  code  dictates  whether  numeric  or  alphabetic  codes 
are  to  be  output  for  a  data  field.    The  next  step  examines  the  type  code  and 
translates  the  input  data  from  alphabetic  t>  numeric  if  so  indicated.  The 
field  is  then  validated  for  range  of  response,  recording  anything  outside  of 
that  range  to  the  audit  file.    The  field  type  code  is  used  by  the  progrin  tc 
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make  a  further  distinction  among  open-ended  item  scores  and  other  numeric  data 
fields.    If  the  data  field  is  an  open-ended  item,  the  routine  uses  the  passed 
value  of  the  number  of  scorers  to  determine  whether  a  score  should  be  marked. 
If  no  scorer  codes  are  indicated  and  the  item  is  marked,  or  a  scorer  code  was 
gridded  and  the  item  is  not  marked,  the  disparity  is  notad  in  th**  audit  file. 
The  routine  then  look^  ahead  to  the  next  field  for  a  secondary  scaring  and 
compares  its  presence  against  the  absence  of  a  second  scorer  code,  and  vice 
versa,  and  again  records  a  disparity  in  the  audit  file.    Moving  the  translated 
and  edited  data  field  into  the  output  buffer  is  the  last  task  performed  in 
this  phase  of  processing. 

The  routine  passes  the  edited  data  string  back  to  the  program,  which 
then  appends  it  to  the  current  output  buffer  and  sets  up  to  process  the  next 
block  within  the  booklet.    The  completed  string  of  data  is  written  to  the  da  a 
file,  using  the  batch  sequence  number  as  the  key  for  direct  access  by  the 
entry  system  programs. 

When  che  next  batch  header  record  or  end  of  file  is  encountered,  the 
program  closes  the  data  end  audit  files,  generates  an  audit  listing,  and 
writes  a  count  of  the  number  of  records  processed  to  the  message  log.  The 
program  then  updates  the  tracking  record  for  that  batch  with  the  current  date 
and  time  and  the  record  count,  and  rewrites  the  record  to  the  tracking  file. 

When  the  program  encounters  the  end  of  the  tape  data  file,  it  closes  and 
rewinds  the  tape  file,  closes  the  tracking  file,  ani  transmits  the  message 
log,  the  audit  listing,  and  the  batch  listing  to  the  printer. 

6.3.3    DATA  ENTRY  AND  RESOLUTION  SYSTEM 

The  d£  a  entry  and  resolution  system  is  essentially  the  same  system  as 
that  used  in  the  1986  assessment,  modified  to  accommodate  changes  in  the 
assessment  design  and  data  entr>  operations.    The  modified  system  must  be  able 
to  process  the  materials  from  three  age  groups  simultaneously,  accommodate  the 
separation  and  merging  of  scannable  booklets  and  answer  sheets,  and  permit  the 
loading  of  excluded  student  questionnaire  data  from  scanning  tapes. 

The  system  comprised  separate  programs  for  each  main  function  (school 
worksheet  entry,  student  data  entry/resolution,  and  questionnaire 
entry/resolution) .    This  separation  permits  the  modification  or  enhancement  of 
one  component  while  allowing  the  others  to  operate.    Access  to  these  programs 
Is  controlled  through  a  menu- type  procedure  written  in  the  VAX  command 
language  and  using  screen  control  directives. 

The  use  of  batch  identification  codes  instead  of  PSU/school/session 
codes  and  of  batch  sequence  numbers  for  student  identification  codes  as  index 
keys  for  the  tracking  and  data  files,  respectively,  greatly  facilitates  the 
management  of  the  system  and  correction  of  incorrectly  gridded  or  keyed 
information. 

Another  addition  to  the  batch  data  records  is  the  data  entry  status 
codes.    The  records  in  a  batch  file  are  generated  in  one  of  two  ways:  the  tape 
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loading  program  or  the  manual  entry  of  the  booklets  rejected  by  the  sc  inner. 
The  manually  entered  records  have  to  undergo  the  two-step  entry  and 
verification  processes,    because  of  the  high  accuracy  rate  of  the  optical 
scanner,  the  loaded  records  are  treated  by  the  system  as  if  they  had  undergont; 
verification.     The  entry  status  code  is  used  to  distinguish  between  records 
that  are  undergoing  manual  processing  and  those  that  were  loaded.    The  code  on 
each  record  is  tested  and  set  by  the  different  prorssses:  entry,  verification, 
loading,  and  resolution. 

The  form  parameters,  which  control  processing  of  each  data  entry  screen, 
are  maintained  in  a  text  library.     Each  set  of  parameters  for  each  form  are 
stored  as  a  separate  member  or  subfile  within  this  library.    This  format 
permits  easy  extraction,  modification,  and  replacement  of  parameter 
information  as  well  as  faster  acc£  s  by  the  entry  programs.    *    .et  of  programs 
was  developed  to  facilitate  the  -^ntry,  documentation,  and  editing  of  the  form 
parameter  data. 
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Chapter  6.4 
EDITING  DATA 


David  S.  Freund  and  Alfred  M.  Rogers 
Educational  Testing  Service 


The  data  editing  process  is  divided  into  three  separate  stepi,: 
validation,  Identification,  and  correction.    Validation  ensu  es  that  each  data 
value  in  the  computer  file  is  of  the  correct  tjrpe,  falls  within  a  range  or  set 
of  ranges  of  values,  and  is  consistent  with  other  data  values.    All  invalid 
data  values  are  identified  and  located  in  the  raw  data  and  either  corrected  or 
flagged  as  unresolvable  in  the  computer  file. 

The  errors  uncovered  by  the  editing  process  fall  into  two  types;  those 
made  by  the  respondent  (e.g.;  choosing  two  responses  for  a  multiple-choice 
exercise  requiring  only  one  response)  and  those  made  iy  data  entry.  The 
validation  process  reports  both  typss  of  error  with  no  knowledge  of  their 
source.    The  identification  process  determines  the  type  of  each  error.  The 
data  entry  errors  are,  for  the  most  part,  correctable;  the  correct  value  can 
be  determined  from  an  examination  of  the  information  on  the  respondent's 
booklet  or  answer  sheet.    Errors  made  by  the  respondent,  however,  are 
difij-cult,  if  not  impossible,  to  correct.     If  the  intent  of  the  respondent 
cannot  be  determined,  the  error  must  remain  unresolved,  but  must  be  flagged  in 
some  way  to  prevent  incorrect  interpretation  in  the  analysis  and  reporting 
pro>.edures. 


6.4.1    ABSENTEE  DATA 

As  described  in  section  6.1.3,  the  absentee  data  (data  for  those 
students  who  weie  absent  on  the  day  of  the  assessment)  were  transcribed  by  the 
NAEP  data  entry  system  from  the  administration  schedules.    Validation  of  this 
data  consisted  of  matching  the  school  and  session  codes  with  those  i?  he 
tracking  file  and  checking  that  the  sex,  grade,  and  birth  date  codes  v ^re 
wi    in  the  appropriate  ranges  for  age  cohort  and  session  type.    A  further 
check  performed  on  these  files  compared  the  number  of  absentee  records  within 
each  session  against  the  absentee  count  field  on  the  corresponding  tracking 
record. 

The  corrected  file  was  again  processed  by  the  validation  program  to 
ensure  that  all  errors  had  been  fixed  and  that  no  new  problems  were  created  in 
the  process.     If  further  errors  were  uncovered,  the  cycle  of  identifying  the 
records,  correcting  the  errors,  and  validating  the  corrected  file  was  repeated 
until  no  more  errors  were  found.    At  this  point,  the  absentee  file  was  ready 
for  transmittal  to  Westat  for  the  estimation  of  sampling  weights. 
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6*4.2    STUDENT  DATA 


The  use  of  scannable  materials,  first  introduced  in  the  1986  assessment 
in  the  form  of  scannable  booklets   -nd  enhanced  in  1988  to  include  scannable 
answer  sheets,  greatly  improved  the  efficie   ly  and  accuracy  of  the 
transcription  piocess  by  removing  the  possibility  of  human  error.  The 
scanning  machinery  was  programmed  to  detect  the  marKv.d  responses  in  unique  and 
fixed  positions  on  each  page;  erroneous  and  out-of-range  response  codes  could 
not  be  generated. 

On  the  other  hand,  removing  hximan  intervention  as  a  source  of  error  also 
prevented  the  ex  ^rcise  of  hximan  judgm^at  when  more  than  one  mark  was  detected 
for  a  single-response  item.    This  would  commonly  happen  when  a  student  marked 
a  second  response  without  erasing  the  fir     ,  or  when  a  student  misinterpreted 
the  question  as  a  "circle  all  that  apply"     ,pe  response.    Neither  the  human 
eye  nor  the  scanninf,  equipment  can  deteinmine  the  student's  intent  in  such  a 
situation.    However,  the  scanning  program  would  also  return  a  multiple - 
response  code  if  the  student  had  incompletely  erased  the  first  response  or 
inadvertently  made  a  stray  mark  on  one  of  the  ovals  (situations  in  which  the 
human  eye  could  determine  the  intended  response).    Hence  there  were 
proportionally  many  more  multiple- response  codes  produced  by  the  seaming 
process  than  by  the  manual  entry  process. 

Furthermore,  collating  errors  in  t!    printing  of  the  booklets  resulted 

in  both  missing  and  multiple  pages,  which  the  scanning  program  was  unprepared 

to  >andle.    A  new  code  was  used  to  designate  responses  to  items  from  pages 
that  were  missing  or  otherwise  unscannab?,e . 

Every  multiple -response  code  and  unscannable-page  code  had  to  be  checked 
against  the  respondent's  booklet  or  answer  sheet  and,  where  possibl  , 
corrected  by  resolution  operators.    At  the  completion  of  resolution 
processing,  all  of  the  batch  student  data  files  were  moved  to  a  single  master 
file  in  preparation  for  transfer  to  the  IBM  mainframe.     A  second  validation 
was  performed  during  this  spooling  process  to  catch  errors  that  had  slipped 
through  the  entry  system  undetected.    An  editing  program  was  developed  for 
applying  corrections  to  this  master  file,  using  the  same  method  as  was  used 
for  the  data  entry  program.    This  master  file  also  served  as  the  basis  for 
preliminary  descriptive  data  analyses  and  quality  control  checks. 


6.4.3    QUESTIONNAIRE  DATA 

The  data  entry  system  was  used  for  the  entry  of  school    teacher,  and 
excluded  student  questionnaire  data  and  served  as  the  first    ine  of  defense 
against  bad  data.    As  described  above,  all  d  ,ta  values  were  validated  for  type 
and  range  as  they  were  entered  fron  the  data  terminal  keyboard.     Special  codes 
assigned  for  multiple  and  indetenninate  responses  were  recorded  and  reported 
via  the  audit  trail.    The  indetf.rminate  values  were  later  corrected  under  the 
resolution  process. 
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The  questionnaire  files  received  the  same  secondary  validation 
processing  as  the  student  data.     Special  attention  was  given  to  the  "circle 
all  that  apply" -type  items  to  ensure  consistency  in  the  coding  of  responses. 
If  a  respondent  circled  on      r  more  of  the  alternatives,  those  would  be  coded 
"1"  while  the  rest  would  bt  coded  "0";  if  no  alternatives  were  n.arked,  yet  th< 
respondent  had  the  opportunity  to  reply,  all  fields  would  be  coded  "0";  if  no 
alternatives  were  marked  and  tlie  respondent  had  not  reached  the  item  or  wa^. 
instruv.ted  to  skip  it,  all  fields  would  be  coded  as  "no  response." 


6.4,4    PROFESSIONALLY  SCORED  ITEM  DATA 

The  open-ended  primary  and  secondary  trait  reading  and  writing  items 
responses  and  open-ended  mathematics,  science,  civics,  and  U.S.  history  item 
responses  were  read  F.nd  scored  prior  to  scanning  processing.    Their  data 
values  were  subjected  to  the  same  editing  procedures  as  the  multiple -choice 
item  responses.    The  open-ended  holistic  trait  writing  items,  however,  were 
not  scored  until  after  scanning  and  resolution  processing.     It  was  not 
feasible  to  enter  so  few  scores  for  each  booklet  through  the  entry  system,  i,o 
these  data  were  subjected  to  a  separate  entry  and  editing  process. 

The  booklets  that  contained  holistic  writing  items  were  batched  and 
forwarded  to  ETS  key  tntry  systems  where  they  were  entered,  verified,  and 
transcribed  to  magnetic  tape.    The  holistic  scores  and  scorer  ID  nunibt,rs  were 
recorded  by  the  scorers  on  the  back  of  the  booklets.    Twenty  percent  of  the 
booklets  were  subjected  to  a  second  set  of  scores  for  use  in  calculating  rater 
reliabilities.    These  scores  and  the  student  ID  number  and  PSU  and  school 
codes  from  the  front  cover  of  the  booklet  were  entered  by  the  key  entry 
operator.    These  tape  files  were  loaded  onto  the  IBM  computer  system  where 
specially  written  validation  programs  performed  thorough  checks  on  the  data 
values.    When  all  of  the  items  had  been  scored,  entered,  validated,  and 
corrected,  the  data  files  were  merged  with  the  student  database. 


6.^  5  CONCLUSION 


Before  the  NAEP  data  entry  methodology  was  developed,  the  editing 
process  for  any  data  file  proceeded  in  the  same  manner  as  for  the  absentee 
data  and  professionally  scored  computer  items.    The  validation  proces.^  was 
especially  inefficient  because  it  was  performed  after  transcription  ana  often 
by  a  second  party  who  did  not  have  immediate  access  to  the  respondent's 
booklet  or  answer  sheet.     Putting  the  valid'-tion  mechanism  at  the  point  of 
entry  removed  most,  if  not  all,  of  this  inefficiency  by  informing  the  entry 
operator  of  a  possible  keying  error  while  the  respondent's  booklet  or  answer 
sheet  was  accessible. 

The  editing  process  does  not  guarantee  that  all  errors  are  removed  from 
the  data;  only  that  the  invalid,  inconsistent,  or  otherwise  unreasonable 
values  have  been  at  least  identified,  if  not  corrected.     If  a  data  value  has 
been  miskeyed  during  the  entry  process  and  meets  the  validation  criteria,  this 
error  could  persist  through  the  editing  process  to  the  analysis  stage  without 
detection.    The  verification  process  detects  rost  of  these  errois  by  comparing 
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independent  entries  of  the  same  data  and  reporting  disagreements.  The 
likelihood  of  an  error  surviving  verification  is  thus  very  small,  but  still 
present.    A  quality  control  process  must  follow  the  entry  and  editing 
processes  to  ensure  that  the  data  values  in  a  given  record  agree  with  the 
responses  in  the  corresponding  instrument. 
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Chapter  6.5 
QUmTY  C0Nr?OL  OF  NAEP  DATA  ENTRY  FOR  1988 


John  J .  Ferris 
Educational  Testing  Service 


Gr  at  care  was  taken  to  control  the  NAEP  data  entry  process  and  the 
quality     "  the  data  entered  thereby.    The  result  was  an  extremely  high  quality 
database >  that  is,  one  with  extremely  low  error  rates.    It  is  of  course 
necessary  to  establish  the  quality  of  any  data  that  will  be  analyzed,  since 
the  most  thorough  and  sophisticated  analyses  of  bad  data  will  yield  nothing  of 
value ►    As  in  past  years,  this  NAEP  database  was  found  to  be  more  than 
accurate  enough  to  support  as  sensitive  an  analysis  as  may  be  desired.     It  is 
worth  noting  that  the  i  nalyses  done  with  the  NAEP  data  ere  only  intended  to 
apply  to  groups  of  respondents,  rather  than  to  individuals,  the  fact  that  such 
analyses  tend  to  be  more  tolerant  of  data  errors  further  enhances  the  already 
high  quality  of  the  NAEP  database. 

The  purpose  of  the  analysis  reported  in  this  chapter  was  to  assess  the 
quality  of  the  data  resulting  from  the  complete  data  enLiy  syst.  .a,  from  the 
actual  instruments  collected  in  the  field  to  the  final  machine  readable 
database  used  in  the  analyses.    The  process  involved  the  seleccion  of 
instruments  at  random  from  among  chose  returned  from  the  field  and  the 
compari^s^n  of  these  instruments,  character  by  character,  with  their 
representations  in  the  final  database.     In  this  way,  we  were  able  to  measure 
che  error  rates  in  the  data  as  well  as  the  success  of  the  data  entry  system. 

Of  course  the  observed  error  rate  cannot  be  taken  at  face  value.  For 
example,  the  sample  of  school  characteristics  and  policies  questionnaires  that 
happened  to  be  selected  for  close  inspection  contained  no  errors  at  all.  To 
conclude  that  the  entire  school  characteristics  questionnaire  database  is 
therefore  error  free  would  be  an  act  of  extreme  or^timism;  we  may  simply  have 
been  lucky  with  this  particular  random  sample,    what  is  needed  is  an 
indication,  of  how  bad  the    rue  error  rate  might  be  given  what  we  observed. 
Such  an  lndic£»tlon  is  provided  by  confidence  limits.    Confidence  limits 
indicate  how  likely  it  is  that  a  value  will  fail  outside  a  specified  range  of 
values  in  a  specified  context  or  distribution.     In  our  analysis,  the  specified 
range  is  an  error  rate  between  zero  ami  some  maximum  value  beyond  which  we  are 
confident  that  the  true  arror  rate  does  not  lie;  the  specified  context  or 
distribution  turns  out  to  be  the  cumulative  binomial  probability  distribution. 
Ar^  example  should  demonstrate  this  technique: 

Let  us  say  that  1,000  booklets  were  processed,  each  with  100 
characters  of  data  transcribed  for  a  total  of  100,000  characters. 
Let  us  say  further  th  c  five  of  these  characters  were  discovered 
to  be  in  error  in  a  random  sample  of  50  booklets  that  were 
completely  checked;  in  other  words,  five  errors  were  found  in  a 
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♦sample  of  5,000  characters.    The  following  expression  may  be  used 
to  establish  the  probability  that  the  true  error  rate  is  .0025  or 
Isss,  rather  than  the  single-value  estimate  of  the  observed  rate 
of  one  in  a  thousand  (.001): 


(^°°°)  •  .0025^  •  (l-.0025)<5^°^-^>      -  .0147 


This  is  the  sum  of  the  probability  o^  finding  five  errors  plus  the 
probability  of  finding  four  errors  plus.   .   .  etc.   .   .  plus  the 
probability  of  finding  zero  errors  in  a  sample  of  5,000  with  a 
true  error  rate  of  .0025,  that  is,  the  probability  of  finding  five 
or  fewer  errors  by  chance  when  the  true  error  rate  is  .0025. 
Notice  that  we  did  not  use  the  size  of  the  database  in  this 
expression.    Actually,  the  assumption  here  is  that  our  sample  of 
5,000  was  drawn  from  a  database  that  is  infinite.     The  smaller  the 
actual  database  is,  the  more  confidence  we  can  have  in  the 
observed  error  rate;  had  there  been  only  5,000  in  the  total 
database,  our  sample  would  have  included  all  the  data    nd  the 
observed  error  rate  would  have  been  the  true  error  rate.  The 
result  of  the  above  computation  allows  us  to  say,  conservatively » 
tb*t  .C025  is  an  upper  limit  on  thf^  true  error  rate  with  98.53 
p*     ant  (i.e.,  1  -   .0147)  confidence;  that  is,  we  are  quite  sure 
th*.     our  true  error  rate  is  no  larger  than  .0025. 

The  indi'^dual  instrtunents  are  briefly  discussed  in  the  following 
sections  and  a  summary  table  (Table  6.5-1)  gives  the  upper  99.8  percent 
confidence  limits  for  the  error  rates  for  each  of  the  instruments  as  well  as 
sampling  rate  information.    The  confidence  limit  of  99.8  percent  was  selected 
to  make  these  results  comparable  co  those  of  previous  administrations  when  the 
same  limit  was  used. 


6.5.1  STUDENT  DATA 


In  recent  past  assessments,  only  one  each  of  the  various  booklets  was 
sampled  for  this  error  rate  analysis.    Due  to  the  complexity  of  the  current 
assessment,  a  larger  number  of  each  booklet  was  examined.     In  all,  over  300 
booklets  out  of  a  total  of  about  120,000  were  compared  in  detail  wich  the 
final  database.    Across  all  scannable  student  da^a,  onlv  about  1  percent  of 
thf  booklets  or  answer  sheets  could  not  be  scann^^J  rnd  had  to  be  keyed  by 
hand;  we  did  not  attempt  to  sample  this  small  ^roup  o£  b'^oklets  separately  for 
quality  analysis  and  relied  instead  on  the  bridges  to  1984  \,o  assess  our 
keying  operation,  since  these  booklets  (reading  and  v/ri<"in^  booklets  51-56) 
were  entirely  keyed.     In  the  past,  keying  error  rates  for  instruments  designed 
to  be  scanned  have  actually  been  somewhat  better  than  keying  error  rates  for 
instruments  designed  to  be  keyed.    The  summary  table  *ives  the  eri.  r  results 
across  all  three  age  classes;  there  were  no  noticeab-.*i  differences  amc  .g  ap^e 
classes . 


160 

J  78 


ERIC 


Table  6.5-1 

Summary  of  Quality  Control  Error  Analysis  for  NAEP  1988  Data  Entry 


Instrunient/Subsaniple 

Student  Data  -  Main 

Student  Data  -  Reading 
and  Writing  Bridge 

Student  Data  -  Other 
Bridges 

Kxcluded  Student 
Questionnaire 

Teacher  Questionnaire 

School  Characteristics 
and  j^olicies 
Questionnaire 


Entry  Diff. 
type  Books 

Scanned  97 

Keyed  18 

Scanned  15 

Mixed  2 


Keyed 
Keyed 


#  Books 
Sampled 

217 

54 

34 

24 

8 
6 


#  Chars. 
Sampled 

29,172 

9,201 

6,311 

2,064 

3,732 
1,532 


Errors 
3 
13 


1 
0 


Observed 
Error  Rate 

.0001 

.0014 

.0003 

.0010 

.0003 
zero 


Upper  99. 8X 
Confidence  Limit 

.0004 

.0030 

.0017 

.0051 

.0023 
.0041 
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6.5.2  EXCLUDED  STUDENT  QUESTIONNAIRE  DATA 


A  total  of  7,791  questionnaires  was  scanned  in  this  assessment  and  this 
group  was  sampled  at  a  rate  of  about  .25  percent;  the  85C  -questionnaires 
(about  one-tenth  of  all  tt ^  excluded  student  questionnaire  data)  f^om  the  fall 
bridge  were  keyed  and  these  were  sampled  at  double  this  rate.    The  few  errors 
tliat  were  discovered  in  the  scanned  data  were  caused  ly  the  scanning  machine. 
Respondents  who  change  their  answers  do  not  always  e?:ase  to  the  satisfaction 
of  a  scanner,  and  this  caused  an  occasional  misreading  of  a  response. 

6.5.3  TEACHER  QUESTIONNAIRE  DATA 

There  were  1,664  teacher  questionnaires  collected  in  this  assessment. 
In  the  eight  that  were  selected  at  random  for  a  complete  verification,  one 
error  was  discovered. 

6.5.4  SCHOOL  CHARACTERISTICS  AND  POLICIES  QUESTIONNAIRE  DATA 

There  were  1,425  school  characteristics  anJ  policies  questionnaires 
collected  in  this  assessment.    No  errors  were  found  in  the  six  quest^'onnaires 
that  were  checked - 


162 

j8I 


Chapter  6  <  6 
CREATION  OF  THE  1988  NAEP  DATABASE 


D.  7id  S.  Freund  and  Alfred  M.  Rogers 
Educational  Testing  Service 


data  transcription  and  editing  procedures  described  in  Chapter  6.1 
resulted  in  the  generation  of  disk  and  tape  files  containing  rious 
assessment  information.     Before  any  ar  ^lysis  could  begin,  these  fi'^es  had  to 
be  brought  together  into  a  comprehensive,  integrated  database.  Sampling 
weights  were  also  required  in  order  to  ma',     valid  statistical  inferences  rbout 
the  population  from  which  the  assessment  sample  was  drawn. 

This  chapter  describes  the  processes       extracting  sample  information 
for  the  derivation  of  sampling  weights,  and  merging,  or  bringing  together,  the 
many  transcription  files  into  the  NAEP  database. 


6.6.1    EXTRACTING  SAMPLE  DATA  TO  DERIVE  WEIGHTS 

'or  each  grade/age  cohort,  up  to  fovx  sets  of  weights  were  requirea  to 
perform  inferential  analyses:  school  weights,  excluded  student  weights, 
student  weights,  and  te6.cher  weights  (age  17  did  not  include  any  teacher 
data).    Because  of  the  method  of  selecting  teachers,  sampling  weights  could 
not  be  assigned  to  teachers,  but  were  instead  assigned  to  students  who  were 
linked  to  participating  teachers.     (See  Chapter  3  for  more  details.) 

All  of  the  sample  information  was  extracted  from  the  data  files,  edited, 
and  transferred  to  tape  files  for  shipment  to  Westat,  where  the  weights  were 
computed.     (See  Chapter  8  for  details  on  computing  weights.)    The  editing 
process  included  both  the  validation  of  the  data  values  (verification  that 
each  data  value  falls  within  a  range  or  set  of  ranges  of  values,  and  is 
consistent  with  ocner  data  values)  and  frequency  distribution  analyses 
containing  count*;  of  the  number  of  students  assessed  for  each  session  to  be 
compcxred  with  tracking  information  from  the  dal.a  entry  system. 

The  school  s^ple  information,  such  as  PSU  and  school  number,  school 
type,  and  sampling  description  of  community  (SDOC) ,  was  available  to  Westat 
from  the  beginni.ig  of  ths  assessment .    No  other  information  was  required  to 
compute  school  sample  weights. 

The  excluded  student  sample  information  was  extracted  from  the  file  of 
excluded  student  questionnaire  data.    This  information  included  questionnaire 
serial  number,  PSU  and  school  code,  grade,  gender,  birth  date,  race/ethnicity, 
and  a  code  indicating  re:^son  for  exclusion.    All  data  fields  were  taken  from 
the  front  cover  of  each  questionnaire,  except  for  the  tx^.xusion  code,  which 
was  derived  .rom  the  response  to  item  2  ("Why  is  this  student  excluded  from 


the  NAEP  survey?")  of  the  questionnaire.  A  listing  of  the  excluded  student 
questionnaires  that  had  not  been  received  at  ETS  was  included  with  the  Tile 
for  each  grade/age  cohort. 

The  student  sample  infor:.ation  can:?,  from  two  sources:     the  student 
database  and  the  absentee  file  from  the  administration  schedules.  The 
assessed  student  sample  information  included  booklet  serial  number,  PSU  and 
school  code,  grade,  gender,  birth  date,  and  race/ethnicity.     Since  the  absent 
students  were  not  observed  and  not  assigned  an  assessment  booklet,  the  booklet 
serial  number  and  race/ethnicity  information  were  not  part  of  the  absentee 
data. 

The  absentee  file  had  to  be  adjusted  for  makeup  sessions.    The  field 
administration  procedures  required  scheduling  of  makeup  sessions  if  absentee 
rates  exceeded  certain  limits.    The  students  attending  these  makeup  sessions 
were  supposed  to  be  originally  sample  i  students  who  were  absent  for  the 
regular  sessions.    Failure  to  remove  the  makeup  students  from  the  absentee 
file  would  have  resulted  in  incorrect  estimates  of  the  munber  of  studenti>  i^i 
thos3  schools.     The  effect  of  .hese  errors  could  have  been  particularly  acute 
in  the  age  17  s^jnple  where  absentee  rates  were  high  aud  many  schools  required 
makeup  sessions. 

The  first  step  in  the  removal  process  was  to  fdentify  the  students  in 
the  student  file  who  atten^ad  makeup  sessions  in  each  school.    Then,  for  each 
school  and  session  type  (spiral  or  tape),  the  gender,  grade,  and  oirth  dates 
of  the  makeup  students  were  matched  with  those  of  the  absentee  students  in  the 
same  school  and  session  type.    The  absentees  identified  by  perfect  matches 
were  removed  from  the  absentee  file:    For  each  unmatched  makeup  student,  a 
randomly  selected  absentee  was  removed  from  the  file.    Tliis  latter  procedure 
was  necessary  only  for  the  age  .  •  sample  in  only  a  few  of  the  mpny  schools 
that  had  makeup  sersions. 

The  teacher  sample  information  was  extracted  from  the  teacher 
questionnaire  data  file.    The  teacher  identified  up  to  ten  students  who 
participated  in  the  assessment  and  met  the  proper  criteria,  i.e.,  for  the 
grade  4/age  9  cohort,  they  we^re  in  the  fourth  grade  and  part  of  the  focused - 
oiB  readi-'ig  assessment  (booklets  8-14)  and  for  the  grade  8/age  13  cohort,  tney 
were  In  the  eighth  grade  and  took  the  focused- BIB  writing  assessment 
(booklets  1-7).     (See  Chapter  3.7  for  more  details.)    The  information  used  by 
Westat  to  produce  student-based  teacher  weights  included  the  n'SU,  school, 
te:  "heiT  code,  booklet  nuraber,  birth  date,  race/ethnicity,  and  gender  for  each 
identified  student  in  that  teacher's  class. 


6.6.2    MERG^      FILES  INTO  THE  NAEP  DATABASE 

The  transcription  pro::ess  resulted  in  the  generation  of  up  to  five  data 
files  for  each  grade/age  cohort:     one  file  for  each  of  the  three 
questionnaires  (no  teacher  questionnaire  for  age  17),  the  student  lesponse 
data  file  from  the  data  entry  system,  and  the  student  holistic  writing  scores 
trom  professional  scoring  and  key  en^ry.    The  process  of  deriving  sample 
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weights  produced  an  additional  fovjr  files  (three  at  age  17)  of  sampling 
•/eights.     Before  data  analyses  could  be  performed,  these  files  had  to  be 
integrated  into  a  coherent  and  comprehensive  database. 

The  database  ultimately  comprised  up  to  four  files  per  cohort:  school, 
teacher,  excluded  student,  and  student  files.    The  student  file  contained  data 
from  all  student  samples — the  main  assessment,  the  bridge  to  1984,  the  bridge 
to  1986,  and  the  civics  bridge  to  1976  and  1982.    The  school  file  could  be 
linked  to  the  other  three  files  (student,  excluded  student,  and  student  based 
teacher)  through  the  PSU  and  school  codes.    The  student-based  teacher  file 
could  be  linked  to  a  subset  of  the  student  main  sample  through  the  student 
booklet  niamber,  PSU,  school,  and  teacher  codes. 

The  school  file  was  created  by  merging  the  school  questionnaire  file 
with  the  school  weights  file  and  wich  a  file  of  school  variables  supplied  by 
Westat  which  included  demDgraphic  information  about  the  schools  that  was 
originally  collected  by  Quality  Education  Data,  Inc.   (QED) .  The  PSl>  and  school 
code  were  used  as  the  matching  criteria.     Each  record  of  the  resulting  file 
was  formed  by  merging  the  weight  inforipation  with  the  response  data  and  the 
QED  data.    Since  not  all  schools  returned  their  questionnaires  and/or  were 
missing  QED  data,  some  of  the  output  records  contain*  d  only  school  identifying 
inf.  rmation  and  weight  information. 

The  teacher  file  was  generated  from  the  teacher  questionnaire  file. 
Since  the  teacher  weights  were  derived  at  the  student  level,  no  information 
had  to  be  added  to    nf*  questionnaire  data. 

The  excluded  student  file  was  the  result  of  merging  the  excluded  student 
questionnaire  file  with  the  excluded  student  weights  file.  The  booklet  serial 
number  was  used  as  the  matching  criterion. 

The  stud.^nt  d^ta  were  created  in  three  steps,  merging  the  student 
response  data  with  the  student  weights,  the  student-based  teacher  weights,  and 
professionally  scored  holistic  writing  item  scores,  in  that  order.     In  all 
:hree  steps,   the  booklet  serial  number  was  used  as  the  matching  criterion. 
The  merging  of  the  professionally  scored  item  data  was  a  more  complex 
procedure  than  the  others,  because  only  a  subset  of  the  student  dat*.  records 
contained  this  data  and  for  those  records  that  did  contain  data,  the  item 
scores  appeared  in  a  different  location  in  each  booklet. 

Vhen  the  appropriate  files  had  beeri  merged,  the  database  was  ready  for 
analysis.    Any  time  that  new  aata  values,  such  as  plausible  values,  were 
derived  external  to  the  dat.iibas,i,  they  were  added  to  the  relevant  files  using 
the  same  matching  procedvxres  as  described  above.    Tlie  public-use  data  tapes 
files  were  later  generated  from  this  database. 

6.6.3    CREATING  THE  MASTER  CATALOG 

A  critical  part  of  any  database  is  its  processing  control  and 
descriptive  information.    A  central  lepositcry  of  this  information  may  be 
accessed  by  all  analys5>s  and  reporting  programs  to  provide  correct  parameters 
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fo.  processing  the  data  fields  as  well  as  to  provide  consistent  labeling  to 
identify  the  results  of  the  analyses.    The  NAEP  master  catalog  file  was 
designed  and  constructed  to  serve  both  of  these  purposes. 

Each  record  of  the  master  catalog  contains  the  processing,  labeling, 
classification,  and  location  information  for  each  data  field  in  the  NAEP 
database.    The  control  parameters  are  used  by  the  access  routines  in  the 
analysis  programs  to  define  the  manner  in  which  the  data  values  are  to  be 
transformed  and  processed. 

Ail  data  fields  have  a  50-character  label  in  the  catalog  describing  the 
contents  of  the  field  and,  wher*^  applicable,  the  source  of  the  field.  The 
data  fields  with  discrete       c&cegorical  values  (e.g.,  multiple  choice  items 
and  professionally  score  i*.  as,  but  not  weight  fields)  have  additional  label 
fields  in  the  catalog  containing  8-  and  20-character  labels  for  those  values. 

The  classification  area  of  the  catalog  record  contains  distinct  fields 
corresponding  to  predefined  classification  categories  for  the  data  fields. 
Fox  a  given  classification  field,  a  nonblank  value  indicates  the  code  within 
that  classification  category  for  the  data  field.    This  petmits  the  collection 
of  identically  classified  items  or  data  fields  by  performing  a  selection 
process  on  one  or  more  classification  fields  in  the  catalog. 

According  to  the  NAEP  design,  it  is  possible  for  item  data  fields  to 
appear  in  more  than  one  student  sample  and  in  more  than  one  block  within  each 
sample.    The  location  fields  of  the  catalog  record  contain  the  age,  block  and, 
where  applicable,  the  sequence  within  the  block  for  each  appearance  of  th^ 
data  field.     (Fields  such  as  plausible  values  and  weights  would  not  contain 
sequence  numbers  since  these  fields  are  not  pertinent  to  a  given  block.) 

The  master  catalog  file  was  constructed  in  parallel  witt  the  collection 
and  transct%tion  of  the  assessment  data  tc  be  ready  for  use  by  analysis 
programs  when  the  database  was  created.    As  new  data  fieMs  were  derived  and 
added  to  the  database,  their  corresponding  descriptive  and  control  information 
were  entered  into  the  catalog. 

One  of  the  most  important  uses  of  the  master  catalog  was  the  control  of 
the  creation  of  the  public-use  data  tapes  files,  codebooks,  and  file  layouts. 
A  synopsis  of  this  process  is  presented  in  the  next  chapter. 
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NAEP  DATAB.IISE  PRODUCTS 
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The  NAEP  databa;.e  described  to  this  point  serves  primaiily  to  support 
analysis  and  reporting  activities  that  are  directly  related  to  the  NAEP  grant. 
This  database  has  a  singular  structure  and  access  methodology  that  is 
integrated  with  the  NAEP  analysis  and  reporting  programs.    One  of  the 
directives  of  the  NAEP  grant  is  to  provide  secondary  researchers  with  a 
nonproprietary  vei'sion  of  the  database  that  is  portable  to  any  computer 
system.    In  the  event  of  transfer  of  NAEP  to  another  client,  the  grant  further 
requires  ETS  to  provide  a  full  copy  of  the  ii:ternal  database  in  a  format  that 
may  be  installed  on  a  different  computer  system. 

In  fulfillment  of  these  requirements,  ETS  provides  three  sets  of 
database  products:  the  item  information  d-^tabase,  the  restricted-use  data 
files,  and  the  public-use  data  files.    The  contents,  format  and  isage  of  these 
products  are  document:ed  in  the  publications  1  isted  under  the  appropriate 
sections  below. 


6.7.1  THE  ITEM  INFORMATION  DATABASE 

The  NAEP       m  information  database  contains      1  oi  the  descriptive, 
processing,  ai  ge  information  for  every  a5sess.aent  item  developed  and  used 

for  NAEP  since  The  primary  unit  of  this  database  is  the  item.  Each 

NAEP  item  is  associated  with  different  levels  of  information,  including  usage 
across  years  and  age  cohorts,  subject  area  classifications,  response  category 
descriptors,  and  locations  of  response  data  on  public -use  data  files. 

The  item  information  database  is  used  for  a  variety  of  essential  NAEP 
tasks:  providing  statistical  information  to  aid  in  test  construction, 
determining  the  usage  of  items  across  assessment  year-r  and  ages  for  trend  and 
cross-sectional  analyses,  labeling  summary  analyses  and  reports,  and 
organizing  item/,  by  subject  area  classifications  for  scaling  analysis. 

The  creation,  structure,  and  use  of  the  NAEP  item  information  database 
for  all  items  used  up  to  and  including  the  1988  assessment  are  fully 
docuTxented  in  the  NAEP  publications,  A  Guide  to  the  NaEP  Item  Information 
Databasij  (Rogers,  i^arone,  &  Kline,  1990)  and  /.  Primer  for  the  NAEP  Item 
Information  bntahase  (Rogers,  Kline,  Barone,  Mychaj lowycz ,  &  Forer,  1989). 

The  procedures  used  to  create  the  1988  version  of  the  item  information 
database  are  the  same  as  those  documented  in  the  guide.    The  updated  version 
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of  the  guide  also  contains  the  learning  area  classification  categories  for  the 
cognitive  items. 


6.7.2  THE  RESTRICTED-USE  DAxA  FILES 

The  restricted-use  data  files  are  for  the  exclusive  internal  use  of  the 
NAEP  grantee.    They  contain  a  complete  copy  of  the  internal  NAEP  respondent 
database  in  a  structured,  documented,  and  portable  format. 

The  internal  database  is  maintained  in  a  compressed  format  to  conserve 
computing  resources  and  to  increase  analysis  efficiency.    The  access  methods 
developed  fot  this  database  locate  data  fields  dynamically  during  the 
execution  of  analysis  programs.    The  restricted-use  data  files,  on  the  other 
hand,  are  "rectangular"  in  structure;  each  data  field  is  in  the  same  location 
on  every  record  within  a  file.    This  static  data  definition,  while  not 
efficient  from  a  computing  resource  standpoint,  is  much  easier  to  document  and 
is  not  dependent  on  any  computing  mdchinery,  operating  system,  or  data  access 
method* 

The  restricted-use  data  files  serve  several  critical  purposes.  They 
provide  an  archive  for  all  respondr  t  data  collected  and  derived  for  Na:p 
since  1970.    They  ensure  compatibility  of  usage  by  expressing  this  data  in 
consistent,  rectang' "".^r  formats.    Their  portability  greatly  facilitates 
transition  of  the  respon-'  »nt  database  Uo  future  NAEP  contractors.  The 
accompanying  data  file  layouts  and  codebooks  provide  a  standardized, 
comprehensive  reference  source  for  NAEP  staff. 

The  contents  and  formats  of  the  NAEP  restricted-use  data  files  are 
documented  in  the  NAEP  publication  A  Guide  to  the  NAEP  Restricted-use  D<xta 
Files  (Rogers,  Barone,  &  Kline,  1989). 

The  procedures  used  to  create  the  restricted-use  data  files  for  the  1988 
assessment  are  the  same  as  those  used  to  create  the  public-use  data  files. 
Since  the  public -use  data  file  distribution  package  contains  mo^ e  products, 
the  generation  procedures  will  be  described  in  the  following  section. 


6.7.3  THE  PUBLIC -USE  3ATA  FILES 

The  public-use  data  files  are  designed  to  enable  any  researcl  :r  with  an 
interest  in  the  National  Assessment  database  to  perform  secondary  ^  .lysis  on 
the  same  data  as  those  used  at  ETS.    They  differ  from  the  restrictea  ase  data 
files  in  one  important  respect:  all  subregional  identification  information  has 
been  encrypted  or  excluded  in  order  to  maintain  the  confidential it;-  of  the 
states,  schools,  ana  students  who  participated  in  the  assessment. 

The  three  elements  of  the  distribution  package  are  the  data  tapes,  the 
printed  documentation,  and  the  microfiche  copies  of  the  assessment 
instruments.  The  complete  set  of  files  for  each  age  cohort  resides  on  a 
separate  tape.    Each  tape  contains,  for  each  sample  or  instrument,  the  data 
file,  a  file  of  control  statements  that  will  generate  an  SPSS-X  system  file,  a 
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file  of  control  statements  that  will  generate  a  SAS  system  file,  and  a 
machine -readable  catalog  file  containing  control  and  descriptive  information, 
intended  for  the  user  \%ho  does  not  use  either  SAS  or  SPSS-X.    The  printed 
documentation  consists  of  four  volumes:  a  guide  to  the  use  of  the  data  files, 
and  a  set  of  data  file  layouts  s^.d  codebooks  for  each  of  the  three  age  cohorts 
("ie  The  NAEP  19S8  Public-use  Data  Tapes  Version  2.0  User  Guide  [Rogers, 
Kline,  Johnson,  Mislevy,  &  Rust,  1990]). 

The  remainder  of  this  section  will  discuss  some  of  the  issues  raised 
during  the  creation  of  the  data  files  and  summarizes  the  procedures  followed 
in  generating  the  data  files  and  related  materials. 


6.7.3.1         File  Definition 

The  first  issue  addressed  in  tht  production  of  the  public-use  data  files 
was  the  organization  and  format  of  the  data  files.  The  NAEP  database  consists 
of  four  data  files  for  each  grade/age  cohort »  corresponding  to  the  three 
questionnaire  instruments  and  the  student  database,  incorporating  the  main 
sample  and  all  five  bridge  samples.    The  logical  relationship  of  the  data 
files  is  a  three-level  hierarchy,  with  the  six  student  and  the  excluded 
student  samples  at  the  lowest  level;  the  teacher  sample  at  the  next  level, 
with  a  linkage  only  to  the  main  sample;  and  the  school  sample  at  the  tcp,  with 
direct  linkages  to  all  samples.    A  linkage  may  be  viewed  as  a  one- to-many 
mapping  of  the  records  within  two  files.     For  example,  one  school  record  can 
link  to  one  or  more  records  in  the  teacher  file,  and  each  of  these  teacher 
records  can  in  tarn  link  to  one  or  more  records  in  the  main  sample  student 
file. 

Two  organization  schemes  were  considered.    The  first  scheme,  using  the 
concept  of  a  static  linkage,  requires  only  seven  files  corresponding  to  the 
seven  student  samples  at  the  lowest  level  of  the  hierarchy.    All  of  the  data 
from  the  higner-level  s-  oiples  would  be  appended  to  and  repeated  across  as  many 
of  the  lower-level  re'- ^ids  as  dictated  by  the  linkr^es.    Using  the  previous 
example,  each  main  sample  record  would  be  appended  by  its  corresponding 
teacher  record  and  school  record.     This  scheme  places  no  demand  on  the  user  to 
define  the  linkages  since  each  data  record  is  complete,  but,  because  of  its 
larger  record  size,  requires  substantially  more  computer  storage  space. 

The  second  scheme,  employing  a  dynamic  linkage,  requires  these  same 
seven  samples,  but  without  the  appended  teacher  and  school  data.     The  teacher 
and  school  sample  data  vould  reside  in  their  own  files,  with  special  data 
fields  in  all  files  to  facilitate  their  linkage  through  program  control.  This 
approach  is  more  economical  in  computer  resource  utilization  but  assumes  a 
more  sophisticated  user.    Ine  potential  for  savings  in  computer  storage  and 
processing  costs  was  the  overriding  consideration  in  choosing  this  scheme. 

The  teacher  questionnaire  for  the  1988  asse*  sment  contained  one  section 
that  provided  a  direct  link  to  individual  students  in  the  main  sample.  The 
file  generated  for  the  teacher  sample,  therefore,  was  based  on  student-level 
data  for  those  linked  students,  with  the  en::ire  teacher  questionnaire  response 
data  appended.    The  benefits  gained  by  doing  this  are  threefold:  analysis  of 
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ceacher  response  data  can  be  properly  performed  at  the  student  level  and  with 
the  appropriate  sample  weights;  the  student*based  teacher  weight  fields  need 
not  be  present  on  the  main  sample  student  file,  as  in  previous  assessments, 
and  the  user  is  freed  from  programming  linkages  between  the  teacher  and  main 
sample  student  files. 


6.7.3.2         Definition  of  the  Variables 

The  selection  and  arrangement  of  data  field,  or  variable*^,  in  each  file 
was  the  next  issue  addressed.    The  initial  step  in  this  process  was  the 
generation  of  a  file  of  descriptors  of  the  variables  for  each  data  file  to  be 
created.     Each  of  these  LABELS  files  contained  one  record  for  each  variable, 
each  record  containing  the  variable  name,  a  c»iiort  description  of  the  variable, 
and  processing  control  information  to  be  used  by  later  steps  in  the  data 
generation  process.    This  file  could  be  edited  for  deletion  of  variables, 
modification  of  control  parameters,  or  reordering  of  the  variables  within  the 
file. 

The  first  program  in  the  processing  stream,  GENLYT,  produced  a  printed 
layout  for  each  file  from  the  information  in  its  corresponding  LABELS  file. 
These  layouts  were  initially  reviewed  for  the  selection  and  ordering  of  the 
variables.     The  variables  thar.  were  excluded  from  public-use  data  file 
processing  fell  primarily  int^>  two  categories:    nonapplicable  and 
confidential. 

The  nonapplicable  variables  were  found  mostly  in  the  student  database. 
In  the  database  used  for  analysis  and  reporting,  the  bridge  samples  were 
combined  with  the  main  sample.     Therefore,  many  of  the  variables  that  applied 
to  the  main  sample  students  did  not  apply  to  the  bridge  sample  students,  and 
vice  versa.    For  example,   the  teacher  code  and  the  student -based  teacher 
weights  were  used  Tor  the  analysis  of  main  sample  data,  but  wtre  not  used  at 
all  in  the  design  for  the  bridge  samples. 

The  confidential  variables  included  any  descriptor  or  code  that  could  be 
used  to  identify  individual  states,  schools,  or  students  in  the  NAEP  sample. 
The  PSU,  school,  teacher,  and  student  identification  codes  used  internally  by 
ETS  and  Westat  were  "scrambled"  according  to  specific  algorithms  to  obtain  new 
codes  for  use  in  linking  the  files  together.     These  new  codes  were  put  on  the 
tapes  in  lieu  of  the  original  codes. 

Another  confidentiality  issue  arose  for  an  item  for  which  studentr  were 
asked  to  identify  the  state  they  had  lived  in  four  years  prior  to  tYe 
assessment.    A  new  variable  was  created  using  the  student's  response  and 
current  state  residency  information  from  the  PSU  code  to  determine  whether  the 
student  had  lived  in  the  same  state,  the  same  region,  or  a  different  region. 

The  ordering  )f  the  variables  within  the  data  files  followed  a  general 
trend  of  decreasing  likelihood  of  usage.     In  this  order  of  likelihood, 
identification  information  preceded  weights,  scores,  and  other  derived 
variables,  which  were  followed  by  the  response  data.    The  identification 
variables  were  generally  those  on  the  front  covers  of  the  instruments.  The 
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derived  variables  included  the  sampling  weights,  the  IRT  scale  values,  and  the 
variables  that  were  derived  from  the  response  data  or  other  sources  for  the 
purpose  of  reporting.     Tie  response  data  variables  were  .irra:  ^ed  according  to 
their  order  in  the  instrument. 

The  data  ^or  the  main  sample  posed  an  additional  challenge  with  its 
multitude  of  booklet  formats  that  had  to  be  structured  into  a  single,  fixed 
format.    The  most  convenient  and  economical  solution  was  to  arrange  the 
"blocks"  of  item  response  data  in  order  within  subject  areas.     The  responses 
to  the  common  background  questionnaire  preceded  all  other  blocks  in  the  new 
record.    The  remaining  blocks  were  grouped  by  5ubject  area,  each  group 
consisting  of  the  subject  area  background  block  followed  by  the  cognitive 
jlocks  in  numerical  order.    Each  record  from  the  input  student  data  file  was 
reformatted  according  to  its  booklet  number;  the  data  for  its  constituent 
blocks  were  moved  into  t)ieir  assigned  locations  in  th^  output  record.  The 
remaining  data  block  eieas  contained  blank  fields,  signifying  that  the  data 
were  missing  by  design. 

In  order  to  process  and  analyze  the  spiral  sample  data  effectively,  the 
user  must  also  be  able  to  determine,  from  a  given  booklet  record,  which  blocks 
of  item  response  data  v^re  present  and  their  relative  order  in  the  instrument, 
liiis  problem  was  remedied  by  the  creation  of  a  set  of  control  variables,  one 
for  each  block,  which  indicated  not  only  the  presence  or  absence  of  the  block 
but  its  order  in  the  instrxmient.     These  control  variables  were  included  with 
the  derived  variables. 


6, 7,3 .3         l-ata  Definition 

To  enable  the  data  files  to  be  processed  on  any  computer  system  using 
any  procedural  or  programming  language,  it  was  desirable  that  the  data  be 
expressed  in  numeric  format.    This  wa.   possible,  but  not  without  the  adoption 
of  certain  conventions  for  reexpressing  the  data  values. 

As  mentioned  in  Chapter  6.3,  the  responses  to  all  multiple-choice  items 
were  transcribed  and  stored  in  the  database  using  the  letter  codes  printed  in 
the  instruments.    This  scheme  afforded  the  advantage  of  saving  storage  space 
for  i*:ems  with  ten  or  more  response  options,  but  at  the  expense  of  transl?ting 
these  codes  into  their  muneric  equivalents  for  analysis  purposes.  The  response 
data  fields  for  most  of  these  itei  s  would  require  a  simple  alphabetic- to- 
numeric  conversion.    However,  the  data  fields  for  items  with  ten  or  more 
response  choices  would  require  "expansion"  before  the  conversion,  since  the 
numeric  value  would  require  two  column  positions.    One  of  the  processing 
control  parameters  on  the  LABELS  file  indicates  whether  or  not  the  data  field 
is  to  be  expanded  before  conversion  and  output. 

The  Els  database  contained  special  codes  to  indicate  certain  response 
conditio.^s:    "I  don't  know"  response,  multiple  response,  omitted  response, 
not-reach3d  response,  and  unresolvable  response,  which  included  out-of- range 
responses  and  responses  that  were  missing  due  to  errors  in  printing  or 
processing.    The  primary  trait  scores  for  the  reading  es' ^y  and  writing  items 
icluded  additional  special  cod^  >  for  ratings  of  "illegible,"  "off  task,"  and 
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nonrateable  by  the  scorers.    All  of  these  codes  had  to  be  reexpressed  in  a 
consistent  numeric  format. 

The  following  convention  was  adopted  and  used  in  the  designation  of 
these  codes:    The  "I  do'   K  know"  and  nonrateable  response  codes  were  always 
com^erted  to  7;  the  omi     A  response  codes  were  converted  to  8;  the 
"not  reached"  response  codes  were  converted  to  9,  ihe  multiple  response  codes 
were  converted  to  0;  the  "illegible"  codes  were  converted  to  5;  and  the 
*'off  task"  codes  were  converted  to  6.    The  out -of- range  and  missing  responses 
were  coded  as  blank  fields,  corresponding  to  the  "missing  by  design" 
designation. 

This  coding  scheme  created  conflict.*;  for  those  multiple  choice  items 
that  had  seven  or  more  valid  response  options  as  well  as  the  **I  don't  know" 
response  and  for  those  open-ended  items  whose  primary  trait  scoring  guide  had 
five  or  more  categories.    These  data  fields  were  also  expanded  to  accommodate 
the  valid  response  values  and  the  s  ecial  codes.    In  these  cases,  the  special 
codes  were  "extended"  to  fill  the  output  data  field:    The  "1  don't  knew"  and 
nonrate/ible  codes  were  extended  from  7  to  77,  etc. 

The  numeric  variables  on  the  tape  files  were  classified  into  two 
categories:     continuous  and  discrete.    The  continuous  variables  include  the 
weights,  IRT  values,  identification  codes,  and  item  responses  where  counts  or 
percentages  were  requested.    The  discrete  variables  include  those  items  for 
which  each  ntuneric  value  corresponds  to  a  response  category.    The  designation 
of  "discrete"  also  includes  those  derived  variables  to  which  numeric 
classification  categories  have  been  assigned.    The  open-ended  items  wet 
treated  as  a  special  subset  of  the  discrete  variables  and  were  assigned  to  a 
separate  category  to  facilitate  their  identification  in  the  documentation, 

6.7.3.4         Data  File  Layouts 

The  data  file  layouts,  as  mentioned  above,  were  the  first  user  product 
to  be  generated  in  the  public  use  data  files  process.    The  generation  program, 
GENLYT,  used  a  LABELS  file  as  input  and  produced  a  printable  fi         Thr  LAYOUT 
file  is  little  more  than  a  formatted  listing  of  the  LABELS  file. 

Each  lj.ne  of  the  LAYOUT  file  contains  the  following  information  for  a 
single  data  field:     sequence  number,  field  name,  output  column  position,  field 
widt!.,  number  of  decimal  places,  data  type,  value  range,  key  or  correct 
response  value,  a  d  a  short  description  of  the  field.    The  sequence  number  of 
each  field  is  implied  from  its  order  on  the  LABELS  file.    The  field  name  is  an 
8-character  label  for  the  field  that  Is  ':o  be  used  consistently  by  all  public - 
use  data  files  materials  to  refer  to  ehat  field  on  that  file.    The  output 
column  position  is  the  relative  location  of  the  beginning  of  that  field  on 
each  record  for  that  file,  using  bytes  or  characters  as  the  unit  of  measure. 
The  field  width  indice  es  the  number  of  columns  used  in  representing  the  data 
values  for  a  field,  the  field  contains  continuous  numeric  data,  the  value 

under  the  number  of  decimal  places  entry  indicates  how  many  places  to  shift 
the  decimal  point  before  processing  data  values. 
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The  data  type  category  uses  three  codes  to  designate  the  nature  of  the 
data  In  the  field.    Continuous  numeric  data  are  coded  "C";  discrete  numeric 
data  are  coded  "D";  open-ended  item  da  i  are  coded  "0."    Additionally,  the 
discrete  numeric  fields  that  include  "I  don't  know"  response  codes  are  coded 
"DI"  and  the  open-ended  items  that  include  nonrateable  response  codes  are 
coded  "01."    If  the  field  typf^  is  discrete  numeric,  the  value  range  is  listed 
as  the  minimum  and  maximum  permitted  values  separated  by  a  hyphen  to  indicate 
range.    If  the  field  i    a  response  to  a  sccrable  item,  the  correct  option 
value,  or  key,  is  printed.    A  range  of  coixrect  options  was  indicated  for  those 
professionally  scored  .Items  that  were  treated  with  cutoff  scoring  for  IRT 
scaling.    Finally,  «ach  variable  was  further  identified  by  a  50-character 
descriptor. 


6.7.3.5  Data  T^Xe  Catalogs 

The  LABELS  file  contains  sufficient  descriptive  information  for 
generating  a  brief  layout  of  the  data  Zile.     However,  to  generate  a  complete 
codebook  document,  substantially  more  information  about  the  data  is  required 
The  CATALOG  file  provides  most  of  this  information. 

lae  CATALOG  file  is  created  by  the  CATGEN  progrc^m  from  the  LABELS  file 
and  the  1988  master  catalog  file.     Each  record  on  the  LABELS  file  generates  a 
CATALOG  record  by  first  i  itrieving  che  master  catalog  record  corresponding  to 
the  field  name.    The  i^aster  catalog  record  contains  usage,  classif i-^ation,  and 
response  code  information,  prefixed  by  the  positional  information  from  the 
LABELS  file:     fleM  sequence  number,  output  column  position,  and  field  wdth. 

The  information  for  the  response  codes,  also  referred  to  a<;  "loils," 
consists  of  the  valid  data  values  for  tho  discrete  numeric  tie]  a 
20 -character  description  of  each.     The  CATGEN  program  uses  addicx^ual  control 
information  from  the  L  VuS  file  to  determine  if  extra  foils  should  be 
generated  and  saved  with  each  CATALOG  record.    The  first  flag  controls 
generation  of  the  "I  don't  know"  or  nonrateable  foil;  the  second  flag 
regulates  omitted  or  "not  reached"  foil  generation;  and  tho  third  flag  demotes 
the  oossibility  of  multiple  responses  x^r  that  field  and  sets  up  an 
appropriate  foil.    All  of  these  control  parameters,  including  the  expansion 
flag,  may  be  alcered  in  the  Jj^iiELS  file  by  use  of  a  text  editor,  in  order  t 
control  the  generation  of  data  or  descriptive,  information  Tor  any  given  fic»d. 

The  LABELS  file  supplies  control  information  for  many  of  the  subsequent 
public-use  duta  processing  steps.    The  CATALOG  file  provides  detailed 
information  Tor  those  and  other  step, 

6.7.3.6  Codebooks 

The  data  file  codebook  is  a  printed  document  containing  complete 
descriptive  information  for  each  data  field.    Most  of  this  int/rmation 
originates  from  the  CATALOG  file;  the  remaining  data  came  from  two  other 
files:    the  COUNTS  file  and  the  IRT  parameters  file. 
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Each  data  field  receives  wt  least  one  line  of  descriptive  information  in 
the  codebook.     If  the  data  type  is  continuous  numeric,  no  more  detail  is 
given.    If  the  variable  is  discrete  numeric,  the  codebook  lists  the  foil 
codes,  foil  labels,  pnd  frequencies  of  each  value  in  the  data  file. 
Additionally,  if  the  field  represents  an  item  used  in  IRT  scaling,  the 
codebook  lists  the  parameters  used  by  the  scaling  program. 

The  frequency  counts  are  not  available  on  the  catalog  file,  l^ut  must  be 
generated  from  the  data.  The  GENFREQ  program  creates  the  COUNTS  file  using  the 
field  name  to  locate  the  variable  in  the  database,  and  the  foil  values  to 
validate  the  range  of  data  values  for  each  field.    This  program  also  serves  as 
a  check  on  the  completeness  of  the  fc  Lis  in  zhe  CATALOG  file,  as  it  flags  any 
data  values  not  represented  by  a  foil  value  and  label. 

The  IRT  parameter  file  is  linked  to  the  CATALOG  file  through  the  field 
name.    Printing  of  the  IRT  parameters  is  governed  by  a  control  flag  in  the 
classification  section  of  the  CATALOG  record. 

The  LAYOUT  and  CODEBOOK  files  are  written  by  their  respective  generation 
programs  to  print- image  disk  data  files.    Draft  copies  are  printed  and 
distributed  for  review  before  the  production  copy  is  generated.  The 
production  copy  is  printed  on  an  IBM  3800  printer  that  .tses  laser- imaging 
technology  to  produce  high-quality,  reproducible  dv curaentation. 

6.7^3.7         Control  Statement  Files  for  Statistical  Packages 

An  additional  requirement  of  the  NAEP  grant  is  to  provide,  for  each 
public-use  data  file,  a  file  of  control  statements  each  for  the  SAS  and  SFSS  X 
statistical  systems  that  will  convert  the  raw  data  file  into  the  system  data 
file  for  that  package.     Two  separate  programs,  GENSAS  and  GENSPX,  generate 
these  control  files  using  the  CATALOG  file  as  input. 

Each  of  the  control  files  contains  separate  sections  for  variable 
definition,  variable  labeling,  missir  ,  value  declaration,  value  labeling,  and 
creation  of  scored  variables  from  the  cognitive  items.    The  variable 
definition  section  describes  the  locations  of  the  fields,  by  name,  in  the 
file,  and,  if  applicable,  the  nxjimber  of  decimal  places  or  type  of  data.  The 
variable  label  identifies  each  field  with  a  50-character  description.  The 
nissing  value  section  identifies  values  of  tho >e  variables  that  are  to  be 
treated  as  missing  and  excluded  from  analyses.    The  value  labels  correspond  to 
the  foils  in  the  CATALOG  file.    The  code  values  and  their  descriptors  are 
listed  for  each  discrete  numeric  variable.    The  scoring  section  is  provided  to 
permit  the  user  to  generate  item  score  variables  in  addition  to  the  item 
response  variables. 

Each  of  the  code  generation  programs  combines  th^ee  steps  into  one 
complex  procedure.     As  each  CATALOG  file  record  is  read,  it  is  broken  into 
several  component  records  according  to  the  information  to  be  used  in  each  of 
the  resultant  sections.    These  record  fragments  are  tagged  with  the  field 
sequence  number  and  a  section  sequence  code.     They  are  th^n  sorted  by  section 
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code  and  sequence  number.  Fincl^y,  the  reorganized  information  is  output  in  a 
structured  format  dictated  by  the  syntax  of  the  processing  language. 

The  generation  of  the  system  files  accomplishes  the  testing  of  these 
control  statement  files.    The  system  files  are  saved  for  use  by  NAEP  staff. 
These  control  statement  files  are  included  on  the  distributed  data  tape  to 
permit  users  with  access  to  SAS  and/or  SPSS-X  to  create  their  own  system 
files . 


6 •7.3.8        Hachine-readablc  Catalog  Files 

For  those  NAEP  data  users  who  have  neither  SAS  nor  SPSS-X  capabilities, 
yet  require  processing  control  information  in  a  computer -reac»"^le  format,  the 
distribution  tape  also  contains  machine-readable  catalog  fi^t^s.  Esch 
machine-readable  catalog  record  contains  processing  control  information,  IRT 
parameters  and  foil  codes  and  labels. 
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Chapter  7 

OVERVIEW  OF  PART  II:    THE  ANALYSIS  OF  1988  NAEP  DATA^ 

Rebecca  Zwick 
Educational  Testing  Ser^/ice 


In  1988,  NAEP  conducted  major  assessments  of  reading,  writing,  civics, 
and  U.S.  history.    In  each  of  these  areas,  the  analyses  included  trend  results 
providing  links  to  previous  assessments,  as  well  as  cross-sectional  results 
for  the  1988  assessment  year,  providing  detailed  information  about  student 
proficiency  for  grades  4,  8,  and  12. 

Another  major  component  of  the  analyses  of  the  1988  data  was  the 
continuing  research  into  the  1986  NAEP  reading  anomaly — the  unexpectedly  low 
reading  proficiency  results  for  ages  9  and  17  that  spurred  a  three-year 
investigation.     For  purposes  of  investigating  the  anomaly  further,  the  1988 
assessment  included  samples  of  students  who  were  assessed  with  1984 
instruments  and  procedures,  as  well  as  samples  who  were  assessed  with  1986 
booklets  and  methods.    The  study  based  on  these  samples,  documented  in  The 
Effect  of  Changes  in  the  National  Assessment:    Disentangling  the  NAEP  1985-86 
Reading  Anomaly  (Beaton  &  Zwick,  1990),  showed  that  seemingly  minor  changes  in 
assessment  technology  had  a  substantial  effect  on  estimated  reading 
proficiency  in  1986.    Because  the  1986  reading  booklets  that  were  administered 
as  part  of  this  study  also  contained  mathematics  and  science  blocks,  these 
data  were  scaled  as  well. 

Finally,  geography  was  assessed  for  grade  12/age  17  only  in  a  special 
3tudy,  CO -sponsored  by  the  National  Geographic  Society. 

7.1      SAMPLES  OF  STUDENTS 

The  samples  of  students  included  in  the  1988  NAEP  assessment  are  listed 
and  described  in  detail  in  Chapter  1.    Only  a  brief  description  of  the  types 
of  1988  samples  is  given  here.    The  1988  samples  were  of  two  general  types: 
bridge  samples,  the  purpose  of  which  was  to  provide  links  to  earlier 
assessments,  and  main  NAEP  sampler? ,  which  were  based  on  a  common  set  of 
assessment  procedures,  incl.iding  wiv^ter  and  spring  administration  times  a-.a 
calendar-year  age  definitions. 

The  1988  bridge  assessments  consisted  of  a  bridge  to  1984  in  reading  and 
writing,  a  bridge  to  1986  in  reading,  mathematics,  science,  and  U.S.  history, 
and  a  bridge  to  1976  and  1982  in  civics.    The  1988  main  NAEP  samples  fell  into 


^Robert  Mislevy  and  Norma  Norris  provided  helpful  comments  on  this 
chapter. 
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threa  categories:    focused-BIB,  intercorrelation,  and  special  studies.  The 
focused-BIB  design  provides  for  booklets  that  include  three  blocks  of  items  in 
a  single  subject  area,  as  well  as  background  items.     Focused-BIB  assessments 
were  conducted  for  reading,  writing,  civics,  and  U.S.  history,  and,  for  grade 
12/age  17,  geography.    The  intercorrelation  samples  received  booklets  that 
included  more  than  one  subject  area  to  allow  researchers  to  investigate  the 
association  between  proficiencies  in  different  subjects.    The  intercorrelation 
booklets  included  blocks  of  reading,  civics,  and  U.S.  history  items  at  all 
three  grade/age  levels,  as  well  as  geography  items  at  grade  12/age  17.  Some 
additional  booklets  were  included  in  the  1988  main  assessment  for  special 
studies.    The  long  writing  booklets  at  all  grade/age  levels  and  the  document 
literacy  booklets  at  the  two  upper  grade/age  levels  fall  into  this  category. 


7.2      ANALYSIS  STEPS 

The  analysis  methods  described  in  the  following  chapters  are  not 
identical  across  subject  areas.     Procedures  depend  on  whether  data  are 
dichotomous  or  ordinal  and  whether  links  across  age  groups  or  across 
assessments  are  required.    Nevertheless,  certain    asic  procedures  are  common 
to  most  or  all  of  the  analyses  described  in  the  following  chapters;  these  are 
summarized  here. 


7.2.1  Item  Analysis 

The  first  analysis  step  in  each  subject  area  was  to  conduct  item 
analyses  within  each  grade/age  cohort  and  within  major  reporting  categories. 
These  preliminary  analyses  had  multiple  purposes:     to  check  the  number  of 
respondents,  the  scoring  of  items,  and  the  coding  of  background  data;  to 
investigate  the  difficulty  level  of  items  and  their  ability  to  di:5tinguish 
between  students  of  high  and  low  proficiency;  to  check  for  speededness;  and  to 
call  attention  to  items  that  may  have  had  popular  but  incorrect  response 
options  (indicating  possible  flaws  in  wording  or  scoring). 

For  each  NAEP  background  l*-em,  th*^  unweighted  and  weighted  percent  of 
students  who  gave  each  response  were  examined,  along  with  the  percent  of 
students  who  omitted  the  item  and  the  percent  who  did  not  reach  the  item.  The 
number  of  respondents  was  also  tabulated.     Each  block  of  dichotomously  scored 
cognitive  items  was  subjected  to  item  analysis  routines  that  yielded,  for  each 
item,  the  number  of  respondents,  the  percent  of  students  who  selected  the 
correct  response  and  each  incorrect  response,  the  percent  who  omitted  the 
item,  the  percent  who  did  not  reach  the  item,  and  the  correlation  between  the 
item  score  and  the  block  score.     In  addition,  summary  statistics  were  comp'  ted 
for  each  block,  including  the  reliability  (internal  consistency). 
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7,2.2  Analyses  of  Differential  Item  FuTiCtioning  Across  *ge,  Gender,  and 

Racial/ethnic  Categories 

For  subject  areas  that  yielded  dichotomous  item  responses,  graphical 
techniques  that  are  available  through  the  NAEP's  modification  of  the  BILOC 
computer  program  (Mlslevy  &  Bock,  1982)  were  used  to  determine  whether  it  wa^ 
reasonable  to  assume  a  common  item  response  function  across  age,  gender,  ani 
racial/ethnic  categories.     (In  the  present  context,  on  item  response  function 
is  the  regression  of  a  dichotomous  item  response  on  5.n  unobserved  proficieicy 
variable.    In  NAEP,  this  nonlinear  regression  is  assiuned  to  take  the  three- 
parameter  logistic  form  described  in  Chapter  9.)    The  NAEP  BILOG  program 
produces  plots  that  show  the  estimated  item  response  function  for  a  particulr>- 
sample  (say,  the  three  age  classes  combined).     In  addition,  BILOG  can  plot 
expected  proportions  correct  for  specified  subsamples  (say,  each  of  the  three 
age  classes)  at  several  points  along  the  proficiency  scale  (see  Mislevy  & 
Bock,  1989  for  further  discussion) .    The  expected  proportions  correct  can  then 
be  examined  to  determine  whether  departures  from  the  common  item  response 
function  are  large  or  systematic.    The  same  method  can  be  used  to  check  for 
differential  item  functioning  across  gender  and  racial/ethnic  groups.  Items 
that  functioned  differently  across  groups  were  reviewed  to  determine  whether 
they  should  be  deleted.     In  the  case  of  items  that  function  differently  across 
age  groups,  another  option  is  to  estimate  separate  item  response  functions  for 
each  age  level  (e.g.,  see  section  10.3). 


7.2.3  Scaling 

Unidimensional  scales  based  on  item  response  theory  were  derived  for 
reading,  writing,  civics,  mathematics,  science,  and  geography.     The  NAEP 
methods  use  random  draws  ("plausible  values")  from  estimated  proficiency 
distributions  to  compute  subpopulation  statistics.     Chapter  9  describes  in 
detail  the  theoretical  underpinnings  of  NAEP's  scaling  methods  and  the 
*.equired  estimation  procedures.    Only  the  basic  analysis  steps  are  outlined 
here. 

For  developing  scales  in  the  dichotomously  scored  subject  areas  (all 
areas  except  writing),  the  steps  were  as  follows: 

1)  Use  NAEP's  version  of  the  BIU)G  program^  (Mislevy  &  Bock,  1982)  to 
estimate  the  parameters  of  the  item  response  functions  on  an 
arbitrary  scale,  assuming  the  three -parameter  logistic  model. 

2)  Use  the  M-GROUP  program  (Sheehan,  1985),  which  implements  the 
method  of  Mislevy  (see  Chapter  9  or  Mislevy,   in  press)  to  estimate 
proficiency  distributions  for  each  student  on  an  arbitrary  scale, 


^N.\EP  BILOG  allow*?  students  in  each  of  the  three  age  classes  to  be 
desigi.ated  as  distinc.  populations.    This  is  important  because,  in  NAEP,  item 
sampling  is  not  random  across  age  classes.     In  this  situation,  age  class 
membership  must  be  taKen  into  account  to  obtain  consistent  item  parameter 
estimates  via  marginal  maximum  likelihood  (see  Mislevy  &  Sheehan,  1989). 
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based  on  these  item  parameter  estimates  and  the  student's 
responses  to  cognitive  items  and  background  questions. 

3)  Determine  the  appropriate  metric  for  reporting  the  results  and 
transform  the  results  as  needed. 

4)  Use  random  draws  from  these  proficiency  distributions  ("plausible 
values'*  in  NAEP  terminology)  for  computing  the  statistics  of 
interest,  such  as  means  for  demographic  groups. 


In  the  c^se  of  the  writing  assessment,  which  yielded  ordinal  scores, 
another  scalin^  model,  the  average  response  method  (ARM)  was  applied.  The 
basic  steps  to  be  applied  were  as  follows: 

1)  Estimate  the  means  on  the  writing  exercises  and  intercorrelations 
among  the  exercises. 

2)  Use  linear  regression  theory  to  impute  a  proficiency  distribution 
for  each  student,  based  on  these  estimated  means  and  correlations 
and  on  the  student's  responses  to  the  writing  exercises  and 
background  questions.    Proficiency  in  this  case  is  defined  as  the 
expected  score  on  the  entire  set  of  writing  exercises,  given  the 
responses  to  a  subset  of  these  exercises. 

3)  Use  random  draws  from  these  proficiency  distributions  for 
computing  the  statistics  of  interest,  such  as  means  for 
demographic  groups. 


As  explained  in  Chapter  9,  the  plausible  values  obtained  through  the  IRT  and 
ARM  approaches  are  not  optimal  estimates  of  individual  proficiency;  instead, 
they  serve  as  interaediate  values  to  be  used  in  estimating  subpopulation 
characteristics.    Under  the  assumptions  of  the  scaling  models,  these 
subpopulation  estimates  will  be  consistent,  which  woiild  not  be  true  of 
subpopulation  estimates  obtained  by  aggregating  optimal  estimates  of 
individual  proficiency. 


7.2.4  Scale  Anchoring 

Scale  anchoring  is  a  process  that  NAEP  has  used,  beginning  with  the  1984 
reading  scale,  to  improve  the  utility  of  proficiency  scale  results  by 
providing  a  c. iterion- referenced  interpretation  of  selected  scale  levels.  In 
this  way,  NAEP  can  furth  r  its  goals  of  describing  what  students  know  and  can 
do  and  stimulating  debate  about  whether  these  levels  of  performance  are 
satisf  actoiry. 

In  NAEP's  scale  anchoring  process,  the  first  step  is  to  choose  four  to 
five  scale  points;  to  be  anchored.    For  each  point,  items  are  then  evaluated  as 
potential  anchor  items,  based  on  the  percent  of  correct  responses  among 
students  with  proficiency  levels  at  that  point,  as  well  as  the  corresponding 
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percent  for  the  next  lower  anchor  point.    For  the  anchoring  of  the  1988  U.S. 
history  and  civics  scales,  an  item  was  considered  to  anchor  at  a  particular 
point  if  (1)  the  p^^rcent  of  students  with  proficiency  levels  approximately 
equal  to  that  point  (i.e.,  within  a  25-point  interval  centered  at  the  point) 
who  answered  correctly  was  at  least  65,  (2)  the  percent  of  students  with 
proficiency  levels  approximately  equal  to  the  next  lower  anchor  point  who 
answered  correctly  was  less  than  50,  and  (3)  the  difference  between  the 
percents  in  (1)  and  (2)  was  at  least  30.     (Of  course,  conditions  (2)  and  (3) 
did  not  apply  to  the  lowest  anchor  point.)    After  the  items  that  anchored  at 
each  point  were  determined,  subject  area  experts  chose  from  among  these  the 
items  that  best  characterized  each  point  and  developed  descriptions  of  the 
anchored  proficiency  levels.    The  descriptions  provide  information  about  the 
types  of  skills  that  are  possessed  by  a  large  proportion  of  students  at  that 
anchor  point,  but  are  not  possessed  by  most  students  at  lower  levels.  The 
percents  of  students  at  or  above  each  anchor  level  are  given  in  NAEP  subject- 
area  reports,  along  with  the  exemplar  items  and  scale-point  descriptions. 

For  the  1988  reading  trend  scale  and  the  mathematics  and  science  scales , 
which  had  already  been  anchored  in  the  past,  previously  established  anchor 
points  were  used.     The  process  for  developing  these  points  was  similar  to  that 
described  above,  but  the  anchoring  criteria  differed  somewhat  (see  Beaton, 
1987a  and  Johnson,  1988  for  information  specific  to  the  mathematics  scales  and 
Yamamoto,  1988  for  information  specific  to  the  science  scales). 

7.3      OVERVIEW  OF  CHAPTERS  8  THROUGH  15 

The  remaining  chapters  in  Part  II  of  this  report  are  as  follows: 

Chapter  8:    The  1988  National  Assessment  used  a  stratified  multistage 
probability  sampling  design  that  provided  for  sampling  certain  subpopulations 
at  higher  rates  (see  Chapter  3).     Because  probabilities  of  selection  are  not 
the  same  for  all  assessed  students,  sampling  weights  must  be  used  in  the 
analysis  of  NAEP  data.    Also,  in  NAEP*s  complex  sample,  observations  are  not 
independent.    As  a  result,  conventional  formulas  for  estimating  the  sampling 
variance  of  statistics  are  inappropriate.    Chapter  8  describes  the  weighting 
procedures  and  methods  for  estimating  sampling  variance  that  are  necessitated 
by  NAEP's  sample  design.    Further  detail  on  sampling  and  weighting  procedures 
is  provided  in  The  1988  National  Assessment  of  Educational  Progress — Sampling 
and  Weighting  Procedures,  Final  Report  (Rust,  Bethel,  Burke,  &  Hansen,  1990), 
a  report  prepared  by  Westat,  Inc. ,  the  NAEP  subcontractor  in  charge  of 
sampling « 

Chapter  9:    A  major  NAEP  innovation  introduced  by  ETS  is  the  reporting 
of  subject-area  results  in  terms  of  proficiency  scales.    Scaling  methods  can 
be  used  to  summarize  results  even  when  students  answer  different  subsets  of 
items.    For  purposes  of  siammarizing  dichotomous  item  responses,  NAEP  developed 
scaling  techniques  that  have  their  roots  in  item  response  theory  and  in  the 
theories  of  .imputation  of  missing  data.    For  application  to  ordinal  data,  such 
as  scores  on  the  NAEP  writing  essays  or  responses  to  NAEP  background  items. 


183 

200 


NAEP  developed  the  average  response  method  (ARM),  which  is  based  on  regression 
theory  and  imputation  techniques.    The  ARM  uses  a  multiple  linear  regression 
approach  to  estimate  a  student's  score  on  a  complete  set  of  items,  given 
responses  to  a  subset  of  items.    Chapter  9  describes  these  two  scaling 
techniques,  the  underlying  theory,  and  the  application  of  these  methods  to 
1988  NAEP  data.     Chapter  9  also  includes  a  discussion  of  the  advantages 
achieved  as  a  result  of  the  adoption  in  1988  of  the  focused-BIB  design. 
Administering  three  blocks  of  items  in  a  single  subject  area  produced  more 
precise  estimates  of  individual  student  proficiency  than  those  that  could  be 
obtained  in  1984  and  1986.    As  explained  in  the  chapter,  this  greater 
precision  reduces  potential  biases  in  the  results  of  secondary  analyses  of  the 
NAEP  data.    The  final  section  of  Chapter  9  gives  an  overview  of  the  NAEP 
scales  that  were  deve*  oped  for  the  1988  assessment. 


Chapter  10:    Two  main  components  of  the  1988  reading  analysis  are 
described  in  this  chapter.     First,  the  reading  trend  results  for  che  years 
1971  through  1984  were  extended  to  include  1988  at  ages  9,  13,  and  17.  The 
results  of  the  reading  trend  analysis,  which  include  the  percents  of  students 
at  or  above  the  reading  scale  anchor  points  established  in  1984,  are  reported 
in  The  ReadLig  Report  Card,  1971-88:     Trends  from  the  Nation's  Report  Card 
(Mullis  &  Jenkins,  1990).     In  addition,  a  detailed  cross-sectional  analysis  of 
reading  for  grades  4,  8,  and  12  in  1988  was  conducted,  including  a  study  of 
the  association  between  reading  proficiency  and  student  background  variables. 
At  grade  4,  background  information  and  data  on  instructional  methods  were 
collected  from  teachers  and  the  relation  of  these  variables  to  reading 
proficiency  was  examined.     The  cross- sectional  analyses  are  reported  in 
Le^m^ng  to  Read  in  Our  Nation's  Schools:    Instruction  and  Achievement  in  1988 
at  Grades  4,  8,  and  12  (Langer,  Applebee,  Mullis,  &  Foertsch,  1990). 

Chapter  11:    Like  the  reading  analysis,  the  writing     .alysis  consisted 
of  two  main  components.    The  writing  trend  results,  which  provide  a  link  to 
1984  for  graces  4,  8,  and  11,  are  reported  in    The  Writing  Report  Card,  1984- 
88:    Findings  from  the  Nation's  Report  Card  (Applebee,  Langer,  Mullis,  & 
Jenkins,  1990).    A  detailed  cross-sectional  analysis  of  writing  for  grades  4, 
8,  and  12  in  1988  was  also  conducted,  including  an  examination  of  the 
association  of  writing  skills  with  instructional  techniques,  student 
background  variables,  and  the  amount  of  time  allocated  for  completion  of  the 
exercises.     For  grade  8,  teacher  data  were  collected  and  their  association 
with  writing  proficiency  was  analyzed.     The  cross-sectional  results  are 
reported  in  Learning  to  Write  in  Our  Nation's  Schools:  Instruction  and 
Achievement  in  1988  at  Grades  4,  8,  and  12  (Applebee,  Langer,  Mullis,  Jenkins, 
&  Foertsch,  1990). 


Chapter  12:    The  trend  and  cross -sectional  analyses  of  the  civics  data 
are  detailed  in  Chapter  12.    The  results  of  the  trend  analysis,  which 
provided  links  to  the  1975-76  and  1981-82  assessments  for  ages  13  and  17,  are 
reported  in  The  1988  Civics  Report  Card:    Trends  in  Achievement  from  1976  to 
1988  at  Ages  13  and  17  and  Achievement  in  1988  at  Grades  4,  8,  and  12 
(Anderson,  Jenkins,  Leming,  MacD^nald,  Mullis,  Turner,  &  Wooster,  1990).  A 
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detailed  csross- sectional  analysis  of  civics  for  grades  4,  8,  and  12  in  1988 
was  also  conducted,  including  an  examination  of  the  association  of  civics 
knowledge  with  instructional  techniques  and  student  background  variables.  The 
cross -sectional  results,  which  also  include  the  percents  of  students  at  or 
above  four  anchor  points  that  i^ere  determined  in  1988,  also  appear  in  The  1988 
Civics  Report  Card, 

Chapter  13:    Cross-sectional  analyses  for  U.S.  history,  analogous  to 
those  described  for  the  preceding  subject  areas,  were  conducted  for  grades  4, 
8,  and  12.    The  outcome  of  these  analyses,  which  include  scale  anchoring 
results,  are  reported  in  The  U.S.  History  Report  Card:    The  Achievement  of 
Fourth",  Eighth-,  and  Twelfth-grade  Students  in  1988  and  Trends  from  1986  to 
1988  in  the  Factual  Knowledge  of  High-school  Juniors  (Hamniack,  Hartoonian, 
Howe,  Jenkins,  Levstik,  MacDonald,  Mullis,  &  Owen,  1990).    The  1988  U.S. 
history  assessment  also  included  a  bridge  sample  linking  the  1988  results  to 
those  of  a  special  study  of  U.S.  history  conducted  for  grade  11  in  1986.  For 
grade  i.l,  trend  results  were  obtained  in  terms  of  item  percents  correct,  and 
were  included  in  the  1990  report  along  with  the  cross- sectional  results. 

Chapter  14:    NAEP  assessed  geography  for  the  first  time  in  1988,  Data 
were  collected  from  students  who  were  17  years  old  or  in  grade  12.    Results  of 
the  cross-sectional  analyses  for  grade  12  which  Include  the  outcome  of  scale 
anchoring,  are  reported  in  The  Geography  Learning  of  Hi^h- school  Seniors 
(Allen,  Bettis,  Kurfman,  MacDonald,  Mullis,  &  Salter,  1990). 


Chapter  15:    As  noted  earlier,  mathematics  and  science  icems  were 
included  in  the  1986  booklets  that  were  administered  in  1988  to  allow  an  in- 
depth  study  of  the  1986  reading  anomaly.    Therefore,  a  small-scale  study  of 
mathemt^tics  and  science  trend,  including  the  derivation  of  scale  anchoring 
results,  was  possible  at  ages  9,  13,  and  17.    Chapter  15  describes  the  methods 
used  to' link  the  1988  results  to  those  for  1973,  1982,  and  1986,     Because  the 
mathematics  and  science  analyses  were  conducted  for  the  purpose  of 
illuminating  the  reading  anomaly,  the  analysis  results,  along  with  further 
detail  on  the  analysis  techniques,  appear  in  Yamamoto's  (1990)  chapter  in  The 
Effect  of  Changes  in  the  National  Assessment:    Disentargling  the  NAEP  ''785 -86 
Reading  Anomaly  rather  than  in  a  subject- area  report. 
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WEIGHTING  PROCEDURES  AND  ESTIMATION  OF  SAMPLING  VARIANCE^ 


Eugene  G .  Johnson 


Educational  Testing  Service 


Keith  F.  Rust  and  Morris  H.  Hansen 


Westat ,  Inc . 


As  was  the  case  in  previous  assessments,  the  1988  National  Assessment 
used  a  complex  sample  design  with  the  goal  to  obtain  a  sample  from  which 
estimates  of  population  and  subpopulation  characteristics  could  be  obtained 
with  reasonably  high  precision  (as  measured  by  low  sampling  variability) .  At 
the  same  time,  it  was  necessary  that  the  sample  be  economically  and 
operationally  feasible  to  obtain.    The  resulting  sample  had  certain  properties 
that  had  to  be  taken  into  account  in  the  proper  analysis  of  the  data  from  the 
assessment. 

The  1988  NAEP  sample  was  obtained  through  a  stratified  multistage 
probability  sampling  design  that  included  provisi-)ns  for  sampling  certain 
sabpopulations  at  higher  rates  (see  Chapter  3).    To  account  for  the 
differential  probabilities  of  selection,  and  to  allow  for  adjustments  for 
nonresponse,  each  student  was  assigned  a  sampling  weight.     Section  8.1 
discusses  the  procedures  used  to  derive  these  sampling  weights. 

Another  consequence  of  the  NAEP  sample  design  is  its  effect  on  the 
estimation  of  sampling  variability.    Because  of  the  effects  of  cluster 
selection  (students  within  schools,  schools  within  primary  sampling  units)  and 
because  of  the  effects  of  certain  adjustments  to  the  sampling  weights 
(nonresponse  adjustment  and  poststratif ication) ,  observations  made  on 
different  students  cannot  be  assumed  to  be  independent  of  one  another.    As  a 
result,  ordinary  formulas  for  the  estimation  of  the  variance  of  sample 
statistics,  based  on  assumptions  of  independence,  will  tend  to  underestimate 
the  true  sampling  variability.    Section  8.2  discusses  the  jackknifi^ig 
technique  used  by  NAEP  to  estimate  sampling  variability.     (The  estimation  of 
variability  due  to  imperfect  measurement  of  individual  proficiency  is 
discussed  in  Chapter  9-) 

The  jackknifing  technique  provides  good  quality  estimates  of  sampling 
variability  but  requires  considerable  computations.    Section  8.3  suggests  the 


^The  statistical  programming  for  this  chapter  was  provided  by  '3avid  Freund, 
Bruce  Kaplan  and  Lee  Ann  Held  of  Educational  lasting  Service,  and  Dalia  Kahane 
of  Westat,  Inc. 
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use  of  design  effects,  combined  with  conventional  variance  estimation 
formulas,  as  a  simple  approximation  to  sampling  variability.    The  technique  is 
demonstrated  for  three  types  of  statistics:    simple  item-level  percent-correct 
statistics,  average  proficiency  scores,  and  simple  regression  coefficients. 

Yet  another  effect  of  the  multistage  NALF  sampling  scheme  is  a  reduction 
of  the  degrees  of  freedom  of  variance  estimates,  as  compared  with  directly 
drawing  independent  samples  of  students  (or  of  schools)  without  clustering 
them.    The  degrees  of  freedom  of  a  variance  estimator  provide  information 
about  its  stability:     the  higher  the  degrees  of  freedom,  the  lower  the 
variability  of  the  estimator.     In  a  simple  random  sample,  the  degrees  of 
freedom  of  a  variance  estimate  depend  upon  the  number  of  subjects  and  on  the 
distribution  of  the  variable  under  consideration.     In  the  HAEP  design,  the 
degrees  of  freedom  are  primaxily  a  function  of  the  number  of  primary  sampling 
units  and  the  number  of  strata  in  the  design,  rather  than  the  number  of 
subjects,  and  the  distribution  of  the  variable  under  consideration  has  less 
impact.     Section  8.4  discusses  the  degrees  of  freedom  for  NAEP  jackknife 
variance  estimates. 

Since  the  sample  design  determines  the  derivati^a  of  the  sampling 
weights  and  the  estimation  of  sampling  variability,  it  will  be  helpful  to  note 
the  key  features  of  the  1988  NAEP  sample  design.    A  description  of  the  design 
appears  in  Chapter  3. 

The  1988  sampTe  was  a  multistage  probability  sample  consisting  of  four 
stages  of  selection.    The  fi.  ^.t  stage  of  selection,  the  primary  sampling  units 
(PSUs),  consisti^d  of  counties  or  groups  of  counties.    Th^e  second  stage  of 
selection  consisted  of  elementary  and  secondary  schools.    The  assignment  of 
sessions  to  sampled  schools  comprised  the  third  stage  of  sampling,  and  the 
fourth  stage  involved  the  selection  of  students  within  schools  and  their 
assignment  to  sessions.    The  probabilities  of  selection  of  the  first-stage 
sartpling  units  were  proportional  to  measures  of  their  size,  while  the 
probability  for  subsequent  stages  of  selection  were  such  that  the  overall 
probabilities  of  selection  of  students  were  approximately  uniform,  with 
exceptions  for  certain  subpopulatlons  that  were  oversampled  by  design.  For 
the  main  assessment,  schools  with  relatively  high  concentratio.is  of  Black 
and/or  Hispanic  students  were  deliberately  sampled  at  twice  the  normal  rate  to 
obtain  larger  samples  of  respondents  from  those  subpopulatlons,  in  order  to 
increase  the  precision  in  the  estimation  of  the  characteristics  of  these 
subpopulatlons.     Students  from  schools  with  smaller  numbers  of  eligibles 
received  lower  probabilities  of  selection,  as  a  means  of  enhancing  the  cost 
efficiency  of  the  sample. 

The  1988  main  assessment  includes  three  student  cohorts:     students  who 
were  either  in  the  fourth  grade  or  9  years  old;  students  ^ho  were  either  in 
the  eighth  grade  or  13  years  old;  and  students  who  were  either  in  the  twelfth 
grade  or  17  years  old.    The  main  -assessment  represents  two  overlapping 
samples.    The  first  sample  represents  students  of  specified  grades  (who  could 
be  of  any  age',.    The  second  sample  represents  students  of  specified  ages  (who 
could  be  of  any  grade).     Students  were  age-eligible  if  chey  were  born  in  the 
appropriate  calendar  year  (1978,  1974.  or  1970).    The  main  assessment  of  all 
grade/age  levels  was  conducted  in  the  winter  and  spring  of  1988  and  the  sample 


188 

m 


design  was  such  that  the  students  assessed  in  the  winter  and  the  students 
assessed  in  the  spring  constitute  two  representative  samples  of  the 
population. 

The  full  1988  assessment  also  includes  a  number  of  additional  samples 
designed  to  determine  the  possible  effects  of  changes  in  age  definitions,  time 
of  testing,  and  mode  of  administration  (elimination  of  the  audiotape  used  for 
pacing  the  items) ,  and  to  provide  links  to  the  results  from  previous 
assessments-     Because  the  purpose  of  these  studies  was  to  provide  the 
statistical  linkage  between  the  1988  data  and  data  from  previous  assessments, 
they  are  referred  to  as  bridge  (or  trend)  studies. 

The  full  1988  NAEP  assessment  thus  includes  a  number  of  different 
samples  from  several  populations.    Each  of  these  samples  has  its  own  set  of 
weights  that  are  to  be  used  to  produce  estimates  about  the  characteristics  of 
the  population  addressed  by  the  sample  (the  target  population).    The  various 
samples  and  their  target  populations  are  as  follows: 

The  Main  Samples  of  Students.    These  samples,  one  for  each  of  the  three 
grade/age  combinations,  were  drawn  in  the  winter  aud  spring,  use  the  new  age 
definitions,  and  consist  of  all  students  assessed  in  the  main  assessment.  The 
target  population  for  each  of  these  samples  consists  of  all  students  rho  are 
in  th3  specified  grade/age  combination  who  were  deemed  assessable  by  their 
school. 


Civics  Bridge  to  1976  and  1982.    This  bridge  (trend)  sample  addresses 
the  subject  area  of  civics  and  consists  of  samples  comparable  to  past 
assessments  of  citizenship  and  social  studies  and  so  uses  pre- 1986  definitions 
of  age  and  time  of  testing.    Since  trend  data  have  been  tradi:-ionally 
collected  only  by  age,  grade  sampling  was  unnecessary.    TJie  ':i\ics  bridge 
sample  consists  of  one  booklet  for  age  13  and  one  booklet         age  17. 
Respondents  to  each  booklet  constitute  a  representative  ^'air.i.le  of  the 
population  of  all  students  of  that  age.     Because  there  we  e  no  reusable  civics 
items  from  previous  assessments  of  9-year-olds,  an  age  9  sample  was  not 
needed. 


Bridge  to  1984.    This  bridge  consists  of  trend  samples  comparable  to  the 
1984  main  assessment  and  addresses  the  subject  areas  of  reading  and  writing. 
The  samples  were  collected  by  grade  and  age  foi.  grade  A/age  9,  grade  8/age  13, 
and  grade  11/age  17,  using  the  age  definitions  and  time  of  testing  from  1984. 
Six  assessment  booklets  were  administered  at  each  grade/age.    The  respondents 
to  the  combined  set  of  assigned  booklets  at  a  given  grade/age  constitute  a 
representative  sample  of  the  population  of  students  who  are  of  the  specified 
grade  or  of  the  specified  age.    The  respondents  to  any  one  of  the  booklets 
also  constitute  a  representative  sample. 

Bridge  to  1986,  Ages  9  and  13.    This  bridge  consists  of  samples  for  ages 
9  and  13  comparable  to  those  used  for  the  measurement  of  trends  in  1986.  The 
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samples  were  collected  by  age  only  and  used  the  same  age  definitions  and  time 
of  testing  as  in  and  in  the  1986  bridge  to  1984.    Three  assessment 

booklecs  were  administered  to  each  age  group  and  the  respondents  to  any  one  of 
the  three  booklets  assigned  to  a  given  age  constitute  a  representative  sample 
of  the  population  of  all  students  of  that  age. 

Bridges  to  1986,  Grade  11/ Age  17.    These  bridge  (trend)  samples,  a  U.S. 
history  bridge  and  a  reavUng,  mathematics,  and  science  bridge  (see  Chapter  1 
for  further  details) ,  are  samples  of  grade  11/age  17  students  comparable  to 
the  1986  main  assessment  sample  and  were  selected  and  administered  using  the 
1986  age  definition  and  time  of  testing.    Since  the  age  definition  and  time  of 
testing  also  correspond  to  those  used  in  samples  from  the  1984  and  earlier 
assessments,  the  students  in  these  bridge  samples  are  comparable  to  the 
students  from  these  earlier  assessments.     (However,  the  performance  results 
are  not  directly  comparable  because  the  earlier  assessments  had  paced 
audiotape  administrations.)    Se ^en  assessment  booklets  were  administered  to 
grade  11/age  17  students.    The  administration  of  these  booklets  was  nonpac  i. 
The  respondents  to  the  combined  set  of  seven  booklets  comprise  a 
representative  sample  of  the  grade  11/age  17  population,  as  do  the  respondents 
to  any  one  of  the  booklets. 

For  purposes  of  sampling  and  weighting,  the  assessment  samples  are 
categorized  as  "te.^      or  "spiral"  according  to  whether  or  not  paced  audiotapes 
are  to  be  used  in  the  administration: 

1)  Tape  samples  are  bridge  samples  that  require  audiotape  pacing  in 
the  assessment  (the  civics  bridge  and  the  age  9  and  13  bridges  to 
1986).    For  these  samples,  all  students  within  a  particular 
assessment  session  receive  the  same  booklet  and  are  paced  through 
at  least  part  of  the  booklet  with  an  audiotape.     These  assessment 
sessions  are  accordingly  referred  to  as  tape  sessions.  The 
students  assigned  to  each  distinct  booklet  of  the  tape  samples  are 
treated  as  a  separate  sample  of  the  population  for  weighting. 

2)  Spiral  samples  are  all  main  assessment  samples  and  the  remaining 
bridge  samples.    For  these  samples,  no  audiotape  pacing  was 
employed  and  the  assessment  booklets  presented  to  a  particular 
sample  are  spiraled  through  each  assessment  session  (that  is,  the 
booklets  are  systematically  interspersed  and  assigned  for  testing 
in  that  order).    These  assessment  sessions  are  referred  to  as 
spiral  sessions.    The  combined  set  of  all  students  assigned  to  any 
of  the  booklets  spiraled  together  is  treated  as  a  sample  of  the 
population  for  weighting. 

8.1    DERIVATION  OF  THE  SAMPLE  WEIGHTS 

As  indicated  previously,  NAEP  uses  differential  sampling  rates, 
deliberately  oversampling  certain  subpopulations  to  obtain  larger  samples  of 
respondents  from  those  subgroups,  thereby  enhancing  the  precision  of  estimates 
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of  characteristics  of  these  oversampled  subgroups.    As  a  result  of  the 
over  samp ling,  these  subpopulations,  corresponding  to  students  from  schools 
with  high  concentrations  of  Black  and/or  Hispanic  students,  are 
overrepresented  in  the  sample.    L^w^r  sampling  rates  were  introduced  also  for 
very  small  schools  (those  schools  with  only  1  to  19  eligible  students).  This 
was  done  in  an  approximately  optimum  manner  as  a  means  of  reducing  variances 
per  unit  of  cost.    Appropriate  estimation  of  population  ch  racteristics  must 
take  disproportionate  representation  into  account.    This  is  accomplished  by 
assigning  a  weight  to  each  respondent,  where  the  weights  properly  account  for 
the  sample  design  and  reflect  the  appropriate  proportional  representation  of 
the  various  types  of  individuals  in  the  population. 

The  weighting  procedures  for  1988  included  computing  the  student's  base 
weight,   the  reciprocal  of  the  probability  that  the  student  was  invited  to  a 
particular  session.    These  base  wtTghts  were  adjusted  for  nonresponse  and  then 
subjected  to  a  trimming  algorithm  to  reduce  a  few  excessively  large  weights. 
The  weights  were  further  adjusted  by  a  poststratif ication  procedure  in  an 
effort  to  reduce  the  sampling  error  and  certain  potential  biases  of  estimates 
relating  to  student  populations  corresponding  to  several  subgroups  of  the 
total  population.    Poststratification  was  performed  by  adjusting  the  weights 
of  the  sampled  students  so  that  the  resulting  estimates  of  the  total  number  of 
students  in  a  number  of  specified  subgroups  of  the  population  corresponded  to 
population  totals  based  on  information  from  the  Current  Population  Survey  and 
Census  Bureau  estimates  of  the  population.    The  subpopulations  were  defined  in 
terms  of  race,  ethnicity,  geographic  region,  age,  and  grade. 

The  following  sections  provide  an  overview  of  the  procedures  used  to 
derive  the  sampling  weights.    Further  details  in  the  derivation  of  these 
weights  can  be  found  in  1988  National  Assessment  of  Educational 
Progress — Sampling  and  Weighting  Procedures,  Final  Report,   (Rust,  Bethel, 
Burke,  &  Hansen,  1990). 


8.1.1  Student  Base  Weight 

The  base  weight  assigned  to  a  student  is  the  reciprocal  of  the 
probability  that  the  student  was  invited  to  a  particular  type  of  assessment 
session,  that  is,  a  main  assessment  session  or  a  particular  bridge  assessment 
session.    That  probability  is  the  product  of  four  factors: 

1)  the  probability  that  the  PSI3  was  selected; 

2)  the  conditional  probability,  given  the  PSU,  that  the  school  was 
selected; 

3)  the  conditional  probability,  given  the  sample  of  schools  in  a  PSH, 
that  the  school  was  allocated  the  specified  t3rpe  of  session  (thii> 
component  is  needed  only  for  the  bridge  samples);  and 

4)  the  conditional  probability,  given  the  school,  that  the  student 
was  invited  to  the  specified  type  of  session. 


191 

207 


Thus,  the  base  weight  for  a  student  may  be  expressed  as  the  product 


Wr  -  PSITWT  •  SCHWT  •  SESSWT  •  STUSCHW 


where  PSUWT,  SCHWT,  SESSWT,  and  STUSC3W  rrf,,  respectively,  the  reciprocals  of 
the  preceding  probabilities.    The  SESSWT  cerm  was  included  only  for  the  bridge 
sainples. 

The  season-specific  base  weight  for  a  stuv^ent  is 


where  SSUBWGT  is  the  reciprocal  of  the  probability  that  the  school  attended  by 
the  student  was  selected  for  assessment  in  the  particular  (winter  or  spring) 
season.    Each  school  had  a  probability  of  0.5  of  being  allocated  to  winter  or 
spring.    In  the  case  of  those  certainty  PSUs  that  were  paired,  with  one  member 
assigned  at  random  to  each  season  (see  Chapter  3),  a  ratio  adjustment  was  made 
to  weight  the  given  pair  member  r    the  size  of  the  pair,  based  on  total 
population.    The  adjustment  factor  for  students  from  the  larger  PSU,  SSUBWGT, 
is  somewhat  less  than  2.0,  whila  for  those  from  the  smaller  member,  SSUBWGT  is 
greater  than  2.0. 

The  base  weight  for  a  student  in  a  bridge  sample  is 


where  tJGTBRDG  is  the  reciprocal  of  the  probability  of  selecting  the  student's 
PSU  into  the  bridge  sample ,  given  that  the  PSU  was  selected  for  the  main 
samples. 

Table  B-1  in  Appendix  B  shows  the  distribution  of  base  weights  for  each 
of  the  separate  sessions  conducted  as  part  of  the  1988  assessment.  Tht 
variations  in  probabilities  of  selection,  and  consequently  of  weights,  were 
Introduced  by  design,  either  to  increase  the  effectiveness  of  the  sample  in 
achieving  its  goals  of  reporting  for  various  subpopulations,  or  to  achieve 
increased  efficiency  per  unit  of  cost. 

8.1.2  Adjustment  of  Base  Weights  for  Nonresponse 

The  base  weight  for  a  selected  student  was  arljusted  by  four  nonresponse 
factors.    One  of  these  was  to  adjust  for  noncooperating  schools,  while  the 
second  (used  only  in  the  case  of  bridge  samples)  was  needed  to  adjust  for 
allocated  sessions  that  occasionally  were  not  conducted.     The   :hird  adjustment 
was  needed  to  account  for  those  few  cases  where,  either  inadvertently  or  on 
the  insistence  of  the  school,  only  scudents  in  the  modal  grade  were  given  a 
chance  of  inclusion  in  the  sample.    The  fourth  adjustment  was  neeaed  to  adjust 
for  students  ^ho  were  (or  should  have  been)  invited  to  the  assessment  but  did 


W^e  -  Wr  •  SSUBWGT 


Wrr  -  Wr  •  WGTBRDG 
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not  appear  either  in  the  scheduled  session  or  a  makeup  session.    Thus,  the 
nonresponse  adjtisted  weight  for  a  student  is  of  the  form 

W  -  Wb  •  SCHNRF  •  SESNRF  •  AOENRF  •  STUNRF 

vhere  the  nonresponse  adjustment  factors  SCHNRF,  SESNRF,  AOENRF,  and  STUNRF 
are  computed  as  described  beluw. 

The  season-specific  base  weight  was  similarly  adjusted  for  the  same  four 
types  of  nonresponse. 

8.1.2-1    School  Nonresponse  Adjustment  (SCIEJRF) 

The  school  .nonresponse  adjustment  W8  intended  to  compensate  for  school 
nonresponse  occurring  before  session  asui^.iment.  These  factors  were  computed 
separately  within  a  FSU  (except  in  a  few  cases  where  PSUs  from  similar  strata 
were  combined  to  give  a  more  stable  adjustment  factor)  . 

The  school  nonresponse  adjustment  factor  in  PSU  h,  SCHNRF^,,  is  given  by 

S  SCHUTjii  • 

ieAh 

SCHNR?h  -  

S  SCHWTht  •  Ghi 

ieBh 


the  school  weight  for  school  i  in  PSU  h; 

the  estimated  number  of  grade/age-eligible  students  in 
school  i  in  PSU  h  based  on  QED  data  (for  sessions 
involving  only  age-eligible  students,  the  number  of 
age  eligibles  in  each  school  was  used); 

consists  of  the  original  sample  of  schools 
(cooperating  and  noncooperating  schools,  but  not 
substitutes);  and 

consists  of  all  schools  cooperating  at  the  time  of 
session  allocation  (including  schools  that  were 
substituted  for  noncooperating  schools). 

For  a  substitute  school,  SCHWThi  is  defined  as  the  school  weight  of  the 
originally  selected  school,  while  the  value  of  G  is  taken  from  the  substitute 
school  itself.    In  those  cases  where  PSUs  were  combined,  the  value  of  PSUWT 
(or  PSUWT  »  WGTBRDG  in  the  case  of  bridge  samples)  was  included  in  the 
numerator  and  denominator  of  the  school  nonresponse  adjustment  factor. 
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where 

SCHWThi 

set  Ajj 
set  Bh 


Table  B-2  in  Appendix  B  shows  the  distribution  of  school  nonresponse 
adjustment  factors  for  each  of  the  1988  assessment  sessions. 


8.1.2.2         Session  Nonresponse  Adjustment  (SESNRP) 

The  session  nonrespoa-se  adjustment  was  intended  to  compensate  for  school 
nonresponse  occurring  in  a  few  PSUs  after  session  assignment  in  the  bridge 
samples.    These  factors  were  computed  separately  within  a  PSU,  except  in  cases 
where  PSUs  from  similar  strata  were  combined  to  give  a  more  stable  adjustment 
factor. 

In  PSU  h,  the  session  nonresponse  adjustment  factor  SESNRFj,  was  given  by 


SESNRFj,  - 


S 


s 


SCHWThi  •  SCHNRFhi  •  SESSOTj,!  •  G^i 


SCHWThi  •  SCHNRFhi  •  SESSWTj,!  •  G^i 


where 


SCHWT 


hi 


SCHNRFi 


hi 


SESSWT 


hi 


Ghi 


set  Bj, 


set  Ci, 


the  school  weight  for  school  i  in  PSU  (or  group  of 
PSUs)  h; 

the  school  nonresponse  adjustment  for  school  i  in  PSU 
h; 

the  session  allocation  weight  for  school  i  in  PSU  h; 

the  estimated  number  of  grade/age- eligible  students  in 
s*:hool  i  in  PSU  h  in  the  case  of  spiral  bridge 
seiisions,  and  the  estimated  number  of  age-eligible 
students  in  the  case  of  the  tape  sessions,  to  which 
only  age  eligibles  were  invited  (the  values  of  Gj,i 
were  based  on  QED  data) ; 

consists  of  all  in-scope  schools  '.  located  to  a 
particular  type  of  session  in  PSU  (or  group  of 
PSUs)  h  that  were  to  be  participating  at  the 
time  of  session  allocation;  and 

consists  of  all  schools  allocated  to  the  session  type 
in  PSU  (or  group  of  PSUs)  h  that  ultimately 
participated. 


In  those  cases  where  PSUs  were  combined,  the  value  of  PSUWT  •  WGTBRDG 
was  included  in  both  the  numerator  and  denominator  of  SESKRF.    Table  B-3  in 
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Appendix  B  shows  the  distribution  of  the  session  nonresponse  adjustment  factor 
fox-  each  of  the  1988  bridge  sample  sessions. 

8.1.2.3         Age-only  Eligibles  Nonresponse  Adjustment  (AOENRF) 

In  a  few  schools  in  which  assessments  took  place,  only  those  students  in 
the  modal  grade  were  listed  for  sampling  (see  Chapter  3),  even  though  there 
was  definite  or  very  strong  evidence  that  other  eligible  students  were 
enrolled.    Thus,  an  adjustment  factor  was  needed  to  account  for  the  fact  that, 
although  students  eligible  by  age  alone  (age-only  eligibles)  were  almost 
certainly  enrolled  in  these  schools,  they  were  not  given  a  chance  of  inclusion 
in  the  sample.    These  factors  were  calculated  separately  by  PSU. 

The  school-level  age-only  eligibles  nonresponse  adjustment  factor  in  PSU 
h,  AOENRFj,,  is  given  for  students  not  in  the  modal  grade  by 

S  SCHWThi  •  SCHNRFhi  •  SESSWI^i  •  SESNRFhi  •  AOj^^ 

AOENRFi,    -  i€Ch 

"S  SCHWThi  •  SCHNRFhi  •  SESSWI^i  •  SESNRF^i  •  AO^i 

i€DH 

where 

the  school  weight  for  school  i  in  PSU  h; 

the  school  nonresponse  adjustment  for  school  i  in  PSU 
h; 

the  session  allocation  weight  for  school  i  in  PSU  h 
(bridge  samples  only) ; 

the  session  nonresponse  adjustment  for  school  i  in  PSU 
h  (bridge  samples  only) ; 

the  estimated  number  of  age -only  eligible  students  in 
PSU  h,  school  i,  based  on  PQ  data; 

consists  of  all  schools  allocated  to  the  particular 
session  type  in  PSU  h  that  ultimately  participated; 
and 

consists  of  all  schools  allocated  to  the  particular 
session  type  in  PSU  h,  that  could  be  reasonably 
supposed  to  have  included  age- only  eligible  students 
in  the  assessment,  if  any,  or  that  had  no  age-only 
eligible  students . 

The  value  of  AOENRF^  for  students  in  the  modal  grade  is  given  as  1.0, 
ince  they  were  not  subject  to  this  component  of  nonresponse.    Table  B-4  in 
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SCHWThi 
SCHNRFhi 

SESSWT^i 

SESNRFt.i 

set  Ch 
set  Dh 
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Appendix  B  shows  the  distribution  of  the  age -only  eligible  nonrespoMse 
adjustment  factor  for  each  of  the  1988  assessment  sessions. 

8*1,2,4    Student  Nonresponse  Adjustment  (STUNRF) 

Student  nonresponse  adjustment  factors       e  completed  separately  for 
spiral  sessions  and  for  each  of  the  tape  sessions  within  each  PSU. 

For  spiral  sessions,  the  student  nonresponse  adjustment  was  made 
separately  for  two  classes  of  students  in  PSU  h  by  age  class:    those  in  or 
above  the  modal  grade  for  their  age,  and  those  below.    This  differentiation 
acknowledges  likely  differences  between  students  in  the  two  classes,  both  in 
their  assessed  abilities  and  in  their  likelihood  of  nonresponse.     For  some 
sessions  in  some  PSUs,  these  two  classes  were  combined,  since  one  or  both  was 
too  small  to  form  the  basis  for  an  adjustment  factor*     For  each  class  c  in  PSU 
h,  the  student  nonresponse  adjustment  factor  STUNRF^c  is  computed  by 

S        SCHWThi  •  SCHNRFhi  •  SESSWThi  •  SESNRFhi  •  AOENRF^i  •  STUSCHW^^i  j 
STUNRFj,,  -  AjI, 

SCHWTi^i  •  SCHNRFhi*  SESSWThi  •  SESNRF^i  •  AOENRF^i  •  STUSCHWhij 


the  school  weight  for  school  i  in  PSU  h; 

the  school  nonresponse  adjustment  factor  for  school  i 
in  PSU  h; 

the  session  allocation  weight  for  spiral  sessions  in 
school  i  in  PS  J  h  (bridge  samples  only) ; 

the  session  nonresponse  adjustment  factor  for  spiral 
sessions  in  school  i  in  PSU  h  (bridge  samples  only) ; 

the  age -only  eligibles  nonresponse  adjustment  factor 
in  PSU  'i,  school  i; 

the  within- school  student  weight  for  student  j  in 
school  i  in  PSU  h; 

consists  of  the  students  in  class  c  in  school  i  in  PSU 
h  who  were  invited  to  the  session;  and 

consists  of  the  students  in  class  c  in  school  i  in  PSU 
h  who  were  assessed  in  the  session. 


where 

SCHWThi 
SCHNRFhi 

SESSWThi 

SESNRF,i 

AOENRFhi 

STUSCHWhij 

Set  A^e 

Set  Bh'c 
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The  sttJdent  nonreuponse  adjustment  for  tape  sessions  was  similar,  except 
that  the  adjustment  was  computed  withia  a  PSU  for  each  tape  booklet  across  all 
students  originally  invited  to  the  assessment  for  that  booklet. 

Table  E-5  in  Appendix  b  shows  tho  distribution  of  student  nonresponse 
adjustment  factors  for  each  of  the  1988  assessment  sessions. 

8.1.3  Trinming  of  Weights 

In  a  number  of  cases,  students  were  assigned  relatively  large  weights. 
One  cause  of  large  weights  was  underestimation  of  the  number  of  eligible 
students  in  some  schools  leading  to  inappropriately  low  p-obabilities  of 
selection  for  those  schools.    A  second  major  cause  is  the  presence  of  large 
schools  (high  schools  in  particular)  in  PSUs  with  small  selection 
probabilities.     In  such  cases,  the  maximam  permissible  within- school  sampling 
rate  (determined  by  the  maximum  sample  size  allowed  per  school — see  Chapter  3) 
could  well  be  smaller  than  the  desired  overall  within- PSU  sampling  rate  for 
students.    Large  weights  arose  also  because  very  small  schools  vere,  by 
design,  sampled  with  low  probabilities.    Other  large  weights  arose  as  the 
result  of  high  levels  of  nonresponse  coupled  with  low  to  moderate 
probabilities  of  selection,  and  the  compounding  of  nonresponse  adjustments  at 
various  levels. 

Students  with  notable  large  weights  have  an  unusually  large  impact  on 
estimates  such  as  weighted  means.    Since,  under  some  simplifying  assumptions 
the  variability  in  weights  contributes  to  the  variance  of  an  overall  estimate 
by  an  approximate  factor  1  +  V^,  where       is  the  relative  variance  of  the 
weights,  an  occasional  unusually  large  weight  is  likely  to  produce  large 
sampling  variances  of  the  statistics  of  interest,  especially  when  the  large 
weights  are  associated  with  students  with  atypical  performance 
characteristics . 

To  rediice  this  problem,  a  procedure  of  trimming  a  few  of  the  more 
extreme  weights  to  values  somewhat  closer  to  the  mean  weight  was  applied. 
This  tvimming  can  increase  the  accuracy  of  the  resulting  survey  estimates, 
substantially  reducing      and  hence  the  siimpling  variance  while  introducing  a 
small  bias.    The  trimming  algorithm  was  identical  to  that  used  in  the  1984  and 
1986  assessments  and  had  the  effect,  approximately,  of  trimming  the  weight  of 
any  school  that  contributed  more  than  a  specified  proportion,  f ,  to  the 
estimated  variance  of  the  estimated  number  of  students  eligible  for 
assessment.    The  trimming  was  done  separately  for  the  spiral  assessment  and 
fcr  each  tape  booklet  in  each  of  tbe  bridge  samples.    In  each  case,  the  value 
of  the  prop  -iTtion  T  was  chosen  to  De  10/K,  where  K  was  the  number  of  schools 
in  which  a  specified  assessment  was  conducted.     The  number  of  schools  where 
weights  were  trimmed  was  small,  being  between  0  and  5  in  each  of  the  samples. 
Table  B-6  in  Appendix  B  shows  the  distribution  of  trimming  factors  for  each  of 
the  1?88  assessment  sessions.     From  the  table  it  is  seen  that  the  most  extreme 
trimming  factors  applied  were  of  the  order  of  0.5  to  1.    While  we  have  not 
extensively  examines    -he  potential  magnitude  of  bias  that  might  be  introduced 
from  such  trimming,     .Jed  rn  the  available  evidence  it  seems  reasonable  to 
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conclude  that  such  bias  would  be  quite  small  and  that  the  reduction  of 
variance  would  be  large  enough  to  result  in  a  reduction  in  the  mean  square 
error. 

8.1.4  Posts tratiflcat ion 

As  in  most  sample  surveys,  the  respondent  weights  are  random  variables 
that  are  subject  to  sampling  variability.    Even  if  there  were  no  nonresponse 
the  respondent  weights  would  at  best  provide  unbiased  estimates  of  the  variois 
subgroup  proportions.    However,  since  unbiasedness  refers  to  e/erage 
perfora/kiice  over  a  conceptu&lly  infinite  number  of  replications  of  the 
spnpling,  it  is  unlikely  tha^  any  given  estimate,  based  on  the  achieved 
sample,  will  exactly  equal  the  population  value.    Furthermore,  the  respondent 
weights  have  been  adjusted  for  nonresponse  and  a  few  extreme  weights  have  bean 
reduced  in  size. 

To  reduce  the  mean  squared  error  of  estimates  using  the  sampling 
weights,  these  weights  were  further  adjusted  so  that  estimated  population 
totals  for  a  number  of  specified  subgroups  of  the  population,  based  on  the  sura 
of  weights  of  students  of  the  sp<»cified  type,  were  the  same  as  presumably 
better  estimates  based  on  composites  of  estimates  from  the  1985  and  1986 
Current  Population  Survey  and  1938  population  projections  made  by  the  Census 
Bureau.    This  adjustment,  called  poststratif ication,  is  intended  especially  to 
reduce  the  mean  squared  error  of  estimates  relating  to  student  populations 
that  span  several  subgroups  of  the  population,  and  thus  to  reduce  the  variance 
of  measures  of  changes  over  time  for  such  student  populations. 

8.1.4.1    1988  Poststratif ication  Procedures 

The  poststratif ication  in  1988  was  done  for  each  grade/age  and 
separately  for  each  of  the  spiral  assessments  and  each  of  the  tape 
assessments.    Within  each  grade/age  and  assessment  type  group, 
poststratif ication  adjustment  cells  were  defined  in  terms  of  race,  ethnicity, 
and  NAEP  region  as  shown  in  Table  8-1. 


Table  8-1 

Major  Subgroups  for  Poststratif ication  in  1988 


Subgroup 

Race 

Ethnic itv 

Region* 

1 

White 

Non-Hispanic 

NE 

2 

White 

Non-Hispanic 

SE 

3 

White 

Non-Hispanic 

Central 

A 

White 

Non-Hispanic 

West 

5 

Any 

Hispanic 

Any 

6 

Black 

Non-Hispanic 

Any 

7 

Other 

Non-Hispanic 

Any 

'RsBions  are  the  atae  as  for  stratifinntior  and  reporting  <S80  Chapter  3),  except 
that  all  o£  Virsinia  is  included  in  the  southeast  region  for  poststratiflcation  purposes 
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The  result  is  seven  poststratif icr tion  cells  for  each  tape  session.  For 
the  assessmfi^nts  involving  both  age  and  grade  eligible  students,  each  of  the 
seven  subgroups  was  further  divided  into  two  or  three  eligibility  classes. 
For  age  classes  9  and  13  and  for  the  grade  11/age  17  bridge  sample,  three 
eligibility  classes  were  used: 

a)  students  eligible  by  both  age  and  grade; 

b)  students  eligible  by  age  only; 

c)  students  eligible  by  grade  only. 

For  the  grade  12/age  17  main  assessment  sample,  the  7  subgroups  were  each 
divided  into  two  subclasses: 

a)  students  eligible  by  grade  (of  any  age); 

b)  students  eligible  by  age  only. 

This  variation  in  the  procedure  from  that  used  for  the  other  age  classes  and 
for  the  grade  11/age  17  bridge  was  adopted  because  the  independent  estimates 
of  the  numbers  cf  students  in  the  population  did  not  provide  consistent  data 
on  the  numbers  of  twelfth  grade  students  eligible  only  by  grade  (see  Rust  et 
al.,  1990,  for  further  details). 

Thus,  there  were  7,  14,  or  21  cells  for  poststratif ication.  The 
poststratified  weight  for  each  student  within  a  particular  cell  was  the 
student's  base  weight,  with  adjustments  for  nonresponse  and  trimming,  times  a 
poststratif ication  factor.     For  each  cell,  the  poststratif ication  factor  is  a 
ratio  whose  denominator  is  the  sum  of  the  weights  (after  adjustments  for 
nonresponse  and  trimming)  of  assessed  and  excluded  students,  and  whose 
numerator  is  an  adjusted  estimate  of  the  total  number  of  students  in  the 
population  who  are  members  of  the  cell.     This  estimated  total  was  a  composite 
based  on  the  October  1985  and  1986  Current  Population  Surveys  and  1988 
population  projeccions.    Table  B-7  in  Appendix  B  shows  the  distribution  of 
poststratif ication  factors  for  each  of  the  1988  assessments. 

8.1.4.2         Differences  From  Earlier  Procedures 

The  poststratif ication  procedures  used  in  1988  were  derived  using  an 
approach  similar  to  those  used  in  1984  and  198o,  but  with  major  variations. 
To  make  the  differences  clear,  the  1984  and  the  1986  procedures  will  be 
described. 

The  same  poststratif ication  procedures  were  used  for  both  the  1984  and 
1986  assessments.     For  the  spiral  a«5sessrcents ,  13  subgroups  were  defined  in 
terms  of  race,  ethnicity,  census  region  and  community  size  (SDOC)  as  shown  in 
Table  8-2.    Each  of  the  13  subgroups  was  further  divided  into  three  classes: 

a)  students  eligible  by  both  age  and  grade; 

b)  students  eligible  by  age  only; 

c)  students  eligible  by  grade  only. 
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Table  8-2 

Major  Subgroups  for  Poststratiflcation  in  1986  and  1984 


SuberouD 

Race 

ccnniciuv 

Region 

1 

wnite 

Nonnlspanic 

tit 

1  0 

2 

Wnite 

Nomllspanlc 

Mb 

<      A  ^ 

i 

white 

NonHlspanlc 

oc,  uentrai 

JL ,  Z 

/, 

H 

wnice 

noiuixspanxc 

Q 

5 

White 

NonHispanic 

SE,  Central 

4.  5 

6 

White 

NonHispanic 

West 

1.  2 

7 

White 

NonHispanic 

West 

3.  4.  5 

8 

Any 

Hispanic 

NE,  SE,  Central 

Any 

9 

Any 

Hispanic 

West 

Any 

10 

Black 

NonHispanic 

NE 

Any 

11 

Black 

NonHispanic 

SE 

Any 

12 

Black 

NonHispanic 

Central ,  West 

Any 

13 

Other 

NonHispanic 

Any 

Any 

^^^f  *SDOe  (Sampl©  Description  of  Comanmity)  qategories.  1— Big  City,  2— Fringe  of  Big  City. 

3-"Htdium  City;  * — Soall  Place;  and  5— Extrecao  Rural. 

The  division  of  the  sample  by  major  subgroup  and  grade/age  eligibility 
class  resulted  in  39  poststratification  cells  for  each  age  class.    The  final 
weight  for  a  student  was  the  product  of  the  base  weight  (as  adjusted  for 
nonresponse  and  after  trimming)  and  a  poststratif ' -  tion  factor  whose 
denominator  was  the  sxira  of  those  weights  for  the  c     L  to  which  the  student 
belongs  and  whose  nximerator  was  an  adjusted  estimate  of  the  total  number  of 
students  in  the  cell.    This  adjusted  c itimate  was  a  composite  of  estimates 
from  the  NAEP  sample  and  an  independent  estimate  based  on  projections  based  on 
Current  Population  Survey  estimates  and  Census  preelections.    The  adjusted 
estimfite  was  a  weighted  mean  of  the  two  estimate j,  the  weights  being  inversely 
proportional  to  the  approximate  variances  of  the  NAEP  and  the  independent 
estimates. 

The  sample  of  students  in  each  of  the  tape  assessments  was  much  smaller 
than  the  sample  for  the  spiral  assessments.     Consequently,  some  subgroups  in 
Table  8-2  were  collapsed  for  poststratification  as  follows: 


1.  2                                  6.  7 

3  8.  9 

4  10,  11.  12 

5  13 


Furthermoire,  to  improve  comparability  with  earlier  assessments,  there  was  no 
subdivision  into  age  and  grade  eligibility  classes,  so  that  there  were  eight 
poststratification  cells  for  each  age  class. 
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The  1988  poststratification  procedures  thus  differ  from  those  used  for 
the  1984  and  the  1986  assessments  in  three  ways: 


1)  The  1988  poststrata  totals  incorporate  current  Census  Bureau 
monthly  population  estimates  by  single  years  of  age  by 
race/ethnicity  groups.    Such  monthly  estimates  were  not  available 
at  the  time  of  the  poststratificatior.  of  the  1984  and  1986 
weights.    Furthermore,  the  use  of  these  estimates  eliminated  the 
need  Co  derive  year-to-year  retention  factors  for  age  17  students, 
as  had  been  done  in  the  previous  years.    This  resulted  from  the 
fact  that  the  estimates  of  in- school  eligibles  were  obtained  using 
data  relating  only  to  the  particular  grade  and  age  in  question, 
rather  than  incorporating  projections  from  younger  ages  and  lower 
grades,  as  was  done  in  1984  and  1986. 

2)  For  the  spiral  assessments,  the  'dumber  of  cells  used  in 
poststratification  was  reduced  from  the  39  cells  used  in  1986  and 
1984  to  the  14  or  21  cells  used  in  1988.     For  the  tape 
assessments,  the  number  of  cells  was  reduced  from  eight  to  seven. 
The  poststrata  used  for  1988  vary  substantially  in  mean 
performance  level  and  yet  are  large  enough  to  produce  reasonably 
stable  poststratification  factors.    The  reduction  in  the  number  of 
cells  was  made  to  increase  the  stability  of  nhe  poststratification 
factors  in  an  effort  to  reduce  the  sampling  variance. 

3)  The  1988  poststrata  totals  were  derived  solely  from  Current 
Population  Survey  data  and  Census  Bureau  population  projections 
and,  in  contrast  to  the  method  used  in  previous  years,  placed  less 
reliance  on  trie  data  from  the  1988  NAEP  samples.    NAEP  data  were 
still  used  to  determine  the  proportion  of  students  eligible  by 
both  age  and  grade;  for  the  spiral  samples  (other  than  for 

grade  12). 

The  1988  procedure  was  adopted  in  order  to  speed  up  the  production  of 
the  weights,  since  poststrata  totals  based  only  on  Current  Population  Survey 
and  Census  data  can  be  derived  well  in  advance  of  the  weighting  of  the  data. 

Appendix  E  describes  the  revisions  made  in  the  poststratification 
weights  for  the  main  samples  for  1984  for  grade  4/age  9  and  for  grade  8/age 
13,  also  to  improve  the  accuracy  of  estimates  for  1984  and  the  trend 
measurements  from  1984  to  subsequent  years. 


A  Measure  of  the  Effect  of  Changes  in  Poststratification  Procedures 

It  is  clearly  important  to  ascertain  the  impact  of  these  changes  in 
postatrcicification  on  the  estimates  of  si*,bgr  .up  proficiencies.    In  particular, 
it  is  important  to  establish  that  the  measurement  of  trend  in  subgroup 
proficiencies  is  affected  in  a  minimal  way  by  this  revision  in  procedures. 
The  approach  used  to  ascertain  the  effect  of  the  change  in  poststratification 
procedures  was  to  reweight  the  li^86  samples  according  to  the  new  procedures 
and  then  compare  the  results:  with  the  previous  results.     (This  approach  is 
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considerably  more  cost  and  time  efficient  than  the  alternative  approach  of 
reweighting  the  It^'^B  data  according  to  the  1986  procedures.) 


Tables  8-3,  8-4,  and  8-5  show  the  result  when  the  age-eligible  students 
in  the  trtnd  samples  of  the  i986  assessm'   t  of  reading  are  reweighted  using 
the  new  poststratif ication  factors.    Th^^  first  two  columns  in  ^ach  table 
compare  the  new  procedure  with  the  old  in  terms  of  the  estimated  relative 
frequencies  by  race/ethnicity,  region,  parental  education,  ai.d  grade.  The 
last  two  columns  compare  the  two  procedur*»s  in  terms  of  the  mean  reading 
proficiencies  for  those  subgroups. 

An  examination  of  these  tables  shows  that  the  effect  of  changing  the 
poststratif ication  procedure  on  mean  proficiency  estimates  is  slight:  in  most 
cases,  the  difference  between  the  proficiency  estimates  based  on  the  two 
procedures  is  less  than  one  standard  error  (of  the  mean  proficiency  based  on 
the  old  method)  and  in  every  case  the  difference  is  less  than  1.25  standard 
errors.    The  differences  betwee.*  estimates  based  on  the  two  poststratif ication 
methods  are  well  within  the  fluctuations  to  be  expected  by  chance  in  eit^   ^  ^f 
the  individual  estimates. 

We  note  that  the  standard  errors  of  the  difference  between  the  original 
and  revised  estimates  are  likely  to  be  relatively  small,  because  of  the  high 
degree  of  correlation  between  the  two  sets  of  estimates.    However,  the 
important  aspects  of  the  change  in  the  method  are  the  sizes  of  the  resulting 
differences  in  estimates,  relative  io  the  precision  of  the  estimates 
themselves,  as  discussed  above. 

8.1.5  The  Final  Student-Weight:    The  Full-Sample  Weight  and  the  Season- 

Specific  Final  Weight 

The  f.*~al  weight  &ssi>gned  to  a  student  is  the  student  full-sample 
weight.    This  weight  is  the  student's  base  weight  after  the  application  of  the 
various  adjustinc  .^s  described  above.    The  student  full-sciTiple  weight  was  used 
to  derive  all  estimates  of  population  and  subpopulation  characteristics  that 
have  been  pre.<?ented  in  the  various  NAEP  reports,  including  simple  estimates 
such  as  the  proport'      of  students  of  a  specified  type  who  would  respond  in  a 
certain  way  to  an  item  and  more  complex  estimates  such  as  mean  proficiency 
levels. 

In  addition  to  the  full-sample  weight,  a  student  season-specific  weight 
was  also  derived*    The  season- specif ic  weight  is  to  be  used  for  analyses  that 
are  based  on  data  from  the  main  NAEP  assessments  from  either  the  winter 
administration  or  the  spring  administration  alone.    The  season- specific  weight 
is  the  student's  season-specific  base  weight  adjusted  for  nonresponse, 
subjected  to  the  t;riraming  algorithm,  and  then  poststratif ied.     The  winter  and 
spring  season-specific  i^eights  were  separately  poststratif  ied  to  the  same 
poststrata  total.*?  as  Visre  used  for  the  full-sample  weights.     The  distributions 
of  the  season-sp  .:ific  poststratif  ication  factors  are  summarized  in  Table  B-7 
in  Appendix  B. 
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Table  8-3 


Effect  of  Change  in  Poststratif ication  Procedures  on 
Relative  Frequencies  and  Mean  Reading  Proficiencies,  Age  9,  1986 


Relative  Frequencies 


New 
Procedure 


Old 
Procedure 


Mean  Reading 
Proficiencies 


New 
Procedure 


Old 
Procedure 


Observed  Race/Ethnicity 

White  76.0X(1.0)  76.5X(1.1) 

nack  15.5X(0.5)  14.9X(0.5) 

Hispanic  6. OX (1.1)  6.2X(1.1) 

Other  2.4X(0.5)  2.5X(0.5) 


214.7(1.5) 
186.4(1.6) 
189.0(2.9) 
204.7(6.2)! 


214. 9(  1.3) 

185. 0(  1.6) 

189. 8(  3.3) 

203. 7(  6.6)! 


Region 

Northeast 
Southeast 
Central 
West 


20.7X(1.1) 
25.9X(2.0) 
26.2X(0.9) 
27.2X(1.6) 


21.1X(1.1) 
22.5X(4.7) 
28.6X(4.0) 
27.7X(1.6) 


212.0(3.0) 
205.2(3.2) 
211.7(2.5) 
206.0(3.1) 


212. 3(  2.7) 

202. 5(  2.7)! 

212. 9(  2.7) 

206. 5(  3.0) 


Grade 


<  Modal  Grade 
at  Modal  Grade 
>  Modal  Grade 


34.2X(1.7) 
65.5X(1.7) 
0.3X(0.1) 


33.9X(1.7) 
65.8X(1.7) 
0.3X(0.1) 


188.3(1.2) 
218.9(1.3) 
238.2(8.8) ! 


189. 4(  1.4) 
218. 5(  1.2) 
241.9(11.3)! 


Parental  Education 

Not  Graduated  H  S  4.3X(0.4)  4.2X(0.4) 

Graduated  H  S  16.0X(0.8)  16.4X(0.7) 

Post  H  S  44.7X(1.2)  44.4X(1.2) 


190.1(2.9) 
201.5(1.4) 
219.2(1.4) 


189. 5(  2.8) 
202. 2(  1.9) 
219. 0(  1.3) 


Total 


208.5(1.3) 


208. 6(  1.2) 


Note:    Standard  errors  in  parenthe-cas 

!  Interpret  with  caution— the  sampling  error  cannot  be  accurately  estimated, 
since  the  coefficient  of  variation  of  the  estimated  total  number  of  students  In  the 
subpopulation  exceeds  20  percent. 
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Table  8-4 


Effect  of  Change  in  Poststratif ication  Procedures  on 
Relative  Frequencies  and  Mean  Reading  Proficiencies,  Age  13,  1986 


Relative  Frequencies 


New 
Procedure 


Old 
Procedure 


Mean  Reading 
Proficiencies 


New 
Procedure 


Old 
Procedure 


Observed  Race/Ethnicity 

White  77.3X(0.9)  76.8X(1.0) 

Black  14.4X(0.8)  14.4X(0.9) 

Hispanic  6.1X(1.0)  6.6X(1.1) 

Other  2.2X(0.3)  2.2X(0.3) 


260.3(0.9) 
239.2(1.9) 
242.1(2.6) 
262.3(3.6) 


258.8(1.2) 
239.3.(1.6) 
242.2T3.1) 
263.9(4.1) 


Region 

North- as t 
Southeast 
Central 
West 


23.9X(1.6) 
23.9X(1.9) 
25.6X(0.6) 
26.7X(1.4) 


22.4X(1.6) 
24.72(5.8) 
24.9X(5.0) 
28.0X(1.5) 


259.6(2.2) 
254.3(1.6) 
25^-.. 6(1. 3) 
2:)6.1(1.8) 


258.7(2.1) 
254.8(1.6)! 
250.8(3.6) 
256.0(1.7) 


Grade 


<  Modal  Grade 
at  Modal  Grade 
>  Modal  Grade 


32.3X(1.6) 
67.3X(1.6) 
0.5X(0.1) 


32.7X(2.1) 
66.8X(2.1) 
0.5X(0.1) 


239.3(1.4) 
264.1(1.0) 
279.5(6.5)! 


238.4(1.4) 
263.0(0.9) 
275.8(6.0)! 


Parental  Education 

Not  Graduated  H  S  7.3X(0.5)  7.8X(1.0) 

Graduated  H  S  29.6X(1.3)  30.5X(1.2) 

Post  H  S  54.0X(2.0)  52.3X(2.1) 


245.4(2.2) 
249.8(1.2) 
263.7(1.0) 


244.2(2.9) 
249.3(1.1) 
262.7(0.9) 


Total 


256.2(0.8) 


255.0(1.0) 


Note:     Standard  errors  in  parenthf-es 

i  Interpret  with  caution — the  sampling  error  cannot  be  accurately  estimated, 
since  the  coefficient  of  variation  of  the  estimated  total  nvimber  of  students  in  the 
subpopuiation  exceeds  20  percent. 
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Table  8-5 


Effect  of  Change  in  Poststratification  Procedures  on 
Relative  Frequencies  and  Mean  Reading  Proficiencies,  Age  17,  1986 


Relative  Frequencies 


Mean  Reading 
Proficiencies 


New 

Procedure 


Old 
Procedure 


New 
Procedure 


Old 

Procedure 


Observed  Race/Ethnicity 

White  76.6X(0.4)  78.0%(0.4) 

Black  14.6X(0.2)  13.52(0.2) 

Hispanic  6.42(0.2)  6.22(0.2) 

Other  2.42(0.3)  2.42(0.3) 


290.9(0.9) 
264.9(1.3) 
266.3(2.4) 
274.1(4.1) 


291.4(0.9) 
265.0(1.2) 
267.5(2.1) 
276.0(4.4) 


Region 

Northeast 
Southeast 
Central 
West 


25.42(1.2) 
24.02(0.6) 
26.12(0.6) 
24.52(0.9) 


23.82(0.3) 
21.22(1.4) 
28.42(1.5) 
26.52(0.5) 


291.2(2.0) 
280.0(1.0) 
287.1(2.1) 
281.7(1.4) 


293.1(2.0) 
279.4(1.0) 
288.1(2.1) 
282.7(1.5) 


Grade 


<  Modal  Grade 
at  Modal  Grade 
>  Modal  Grade 


24.92(0.6) 
65.8^(0.4) 
9.32(0.6) 


21.82(0.6) 
70.32(0.4) 
7.92(0.5) 


258.0(0.9) 
293.1(0.8) 
301.2(2.0) 


257.7(1.0) 
293.1(0.8) 
301.0(2.1) 


Parental  Education 

Not  Graduated  H  S  9.32(0.5)  8.92(0.6) 

Graduated  H  S  27.82(0.9)  27.72(0.8) 

Post  H  S  58.92(1.3)  59.42(1.2) 


265.0(1.1) 
274.9(0.8) 
295.3(0.8) 


266.3(1.4) 
275.9(0.8) 
295.8(0.9) 


Total 


285.1(0.8) 


286.0(0.9) 


Note:     Standard  errors  in  parentheses 
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The  effects  of  all  of  the  adjustments  to  the  base  weights  are  summarized 
in  Table  B-8  in  Appendix  B,  which  shows  the  distribution  of  the  single  factor 
given  as  the  product  of  SCHNKF,  SESNRF,  AOtlNRF,  STUNRF,  the  trimming  factor, 
and  the  posts tratification  factor,  for  each  of  the  assessment  components.  The 
distributions  of  the  final  student  weight.,  are  given  in  Table  B-9  in 
Appendix  B. 

As  indicated  earlier,  under  some  simplifying  assumptiois  the  factor 
1  +       indicates  the  approximate  relative  increase  in  variance  of  estLnates 
resulting  from  the  variabilicy  in  the  weights.    The  factor  1  +       for  each 
sample  is  readily  derivable  from  Table  B-9  by  adding  1  to  the  square  of  the 
ratio  of  the  standard  deviation  to  the  mean  weight.    These  factors,  resulting 
from  the  combined  effect  of  the  variations  in  weights  introduced  by  design  and 
from  other  causes,  as  discussed  above,  are  gen'trally  about  1.3  for  the  main 
samples.    They  vary  from  1.12  to  1.53  for  the  various  bridge  samples. 

8.1.6    Other  Weights 

In  addition  to  the  weights  for  the  assesseu  students,  weights  were  also 
derived  for  excluded  students  and  for  the  students  whose  teachers  participated 
in  the  teacher  survey. 

Weights  for  excluded  students.     Excluded  students  are  students  who  were 
designated  by  the  schools  as  unable  to  complete  the  assessment  because  they 
were  non-English  speaking,  mildly  mentally  retarded  (educable),  or 
functionally  disabled.     Since  the  same  grade  and  age  eligibility  definitions 
apply,  no  distinction  is  made  between  students  excluded  from  the  bridge 
assessments  and  students  excluded  from  the  main  assessments  for  grade  4/age  9 
and  grade  8/age  13.    However,  sin^e  the  age  and  grade  eligibility  definitions 
differ  for  the  oldest  age  class,  the  excluded  students  from  the  grade  11/age 
17  bridge  assessments  (with  an  October-September  age  definition  and  modal 
grade  of  11)  are  treated  as  separate  from  the  excluded  students  from  the  grade 
12/age  17  main  assessment  (with  a  calendar-year  age  definition  and  modal  grade 
of  12). 

As  in  the  case  of  the  weights  for  the  assessed  students,  the  excluded 
student  weights  were  constructed  from  components  reflecting  the  probability  of 
selection,  correction  for  nonresponse,  weight  trimming,  and 

poststratification.     Further  details  on  the  derivation  of  the  excluded  student 
weights  can  be  found  in  Rust  et  al.   (1990).    The  distributions  of  the  base 
weights,  the  various  weight  components  and  their  composite,  and  the  final 
student  weight  are  given  in  Table  B-10  in  .Appendix  B  fcr  each  of  the  four 
samples. 

Weights  for  students  in  the  teacher  survey.    The  reading  teachers  of 
every  grade  4  student  who  was  assessed  for  reading  in  the  main  assessment  and 
the  writing  teachers  of  every  grade  8  student  who  was  assessed  for  writing  in 
the  main  assessment  were  identified  within  each  school.    Up  to  seven  of  these 
teachers  within  each  school  were  selected  to  complete  the  teacher 
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questionnaire;  in  schools  with  more  than  seven  such  teachers,  a  sample  of  five 
teachers  of  students  assessed  for  reading  or  writing  (as  appropriate)  in  the 
main  assessment  was  selected.    Every  selected  teacher  was  provided  a  list  of 
all  his  or  her  students  (up  to  a  maximxim  of  10)  who  had  been  assessed  for 
reading  or  writing  (as  appropriate)  in  the  main  assessment.    If  more  than  10 
students  fit  the  criteria,  a  random  sample  of  10  such  students  was  provided. 
The  selected  teachers  were  asked  to  complete  a  detailed  questionnaire  about 
the  capabilities  of  each  selected  student  and  the  kinds  of  reading  or  writing 
instruction  the  student  received. 

These  data  can  be  analyzed  using  the  teacher -student  weights.     It  is 
important  to  note  that  the  teacher -student  weights  are  appropriate  for  use  in 
estimating  the  number  or  percent  of  students  in  the  total  population  who  have 
various  characteristics;  they  are  not  appropriate  for  use  in  estimating  the 
number  of  teachers  or  the  number  of  teachers  with  various  characteristics. 
They  are  supplied  only  for  a  subsample  of  students  anc  -^re  appropriate  for  use 
in  analyses  involving  the  teacher-characterictics  of  students  (for  example,  to 
estimate  the  proportion  of  fourth-grade  students  whose  reading  teachers  h^..e 
at  iec^st  a  master's  degree). 

The  teacher -student  weights  are  based  on  the  final  student  weights  of 
those  students  in  the  main  assessment  who  are  linked  to  a  completed  teacher 
questionnaire.  The  teacher-student  weight  is  the  nonresponse- adjusted  student 
weight  further  adjusted  for  the  probability  that  the  student's  teacher  '  as 
selected  as  well  as  for  nonresponse  on  the  part  of  the  teachers. 
Additionally,  the  teacher -student  weights  were  subjected  to  the  trimming 
algorithm  and  poststratif ication  adjustments.     Further  details  on  the 
construction  of  these  weights  appear  in  Rust  et  al.   (1990).    The  distributions 
of  the  teacher  subsampling  adjustment,  the  teacher  nonresponse  adjustment,  the 
trimming  factor,  the  poststratif ication  factor,  and  their  composite,  together 
with  that  of  the  final  teacher-student  weights,  are  summarized  in  Table  B-11 
in  Appendix  B. 

Finally,  in  addition  to  these  weights,  which  were  used  to  derive  all 
estimates  of  population  and  subpopulation  characteristics,  other  sets  of 
weights,  called  jackknife  replicate  weights,  were  derived  to  facilitate  the 
estimation  of  sampling  variability  by  the  jackknife  variance  estimation 
cecnnique.    These  weights  and  the  jackknife  estimator  are  discussed  in  the 
next  section. 


8.2      PROCEDURES  USED  BY  NAEP  TO  ESTIMATE  SAMPLING  VARIABILITY 

A  major  source  of  uncertainty  in  the  estimation  of  the  value  in  the 
population  of  a  variable  of  interest  exists  because  information  about  the 
variable  is  obtained  on  only  a  sample  from  the  population.    To  reflect  this 
fact,  it  is  important  to  attach  to  any  statistic  (e.g.,  a  mean)  an  estimate  of 
the  sampling  variability  to  be  expected  for  that  statistic.    Estimates  of 
sampling  variability  provide  information  about  how  much  the  value  of  a  given 
statistic  would  be  likely  to  change  if  the  statistic  had  been  based  on 
another,  equivalent,  sample  of  individuals  drawn  in  exactly  the  same  manner  as 
the  achieved  sample. 
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Another  important  source  of  variability  is  that  due  to  imprecision  in 
the  measureirent  of  individual  proficiencies.     In  NAEP,  proficiencies  in 
subject  areas  are  summarized  through  item  response  theory  (IRT)  or  average 
response  method  (ARM)  models,  but  not  in  the  way  that  these  models  are  used  in 
standard  applications  where  each  person  responds  to  enough  items  to  allow  for 
precise  estimation  of  that  person's  proficiency.     In  NAEP,  each  individual 
responds  to  relatively  few  items  so  that  individual  proficiency  values  are  not 
well  determined.    Consequently,  the  variance  of  any  statistic  based  on 
proficiency  values  has  a  component  due  to  the  imprecision  in  the  .measurement 
of  the  proficiencies  of  the  sampled  individuals  in  addition  to  a  component 
measuring  sampling  variability.    The  estimation  of  the  component  of 
variability  due  to  measurement  .mprecision  and  its  effect  on  the  total 
variability  of  statistics  based  on  proficiency  values  are  discussed  in 
Chapter  9. 

The  estimation       the  sampling  variability  of  any  statistic  must  take 
into  account  the  sample  design.     In  particular,  because  of  the  effects  of 
cluster  selection  (students  within  schools,  schools  within  PSUs)  and  because 
of  effects  of  nonresponse  and  poststratif ication  adjustments,  observations 
i^jide  on  different  students  cannot  be  assumed  to  be  independent  of  each  other 
(and  are,  in  fact,  generally  positively  correlated).    Furthermore,  to  account 
for  the  differential  probabilities  of  selection  (and  che  various  adjustments), 
each  student  has  an  associated  samplit^g  weight,  which  should  be  used  in  the 
computation  of  any  statistic  and  which  is  itself  subject  to  sampling 
variability.     Ignoring  the  special  characteristics  of  the  sample  design  and 
treating  the  data  as  if  the  observations  were  independent  and  identically 
distributed,  will  generally  produce  underestimates  of  the  true  sampling 
variability. 

The  proper  estimation  of  the  sampling  variability  of  a  statistic  based 
on  the  NAEP  data  is  complicated  and  requires  techniques  beyond  those  commonly 
available  in  standard  statistical  packages.    Fortunately,  the  Jackknife 
procedure  (see,  e.g.,  Wolcer,  1985;  Frankel,  1971)  provides  good  quality 
estimates  of  the  sampling  variability  of  most  statistics,  at  the  expense  of 
increased  computation,  and  can  be  used  in  concert  with  standard  statistical 
packages  to  obtain  a  proper  estimate  of  sampling  variability. 

The  jackknife  procedure  used  by  NAEP  has  a  number  of  properties  that 
make  it  particularly  suited  for  the  analysis  of  NAEP  data.    When  properly 
applied,  a  jackknife  estimate  of  the  variability  of  a  linear  estimator  (such 
as  a  total)  will  be  the  same  as  the  standard  textbook  variance  estimate 
specified  for  the  sample  design  (if  tho  first-stage  units  were  sampled  with 
replacement  and  approximately  so  otherwise).    Additionally,  if  the  finite 
sampling  corrections  for  the  first  stage  units  can  be  ignored,  the  jackknife 
produces  asymptotically  consistent  variance  estimates  for  statistics  such  as 
ratios,  regression  estimates  or  weighted  means  and  for  any  other  nonlinear 
statistic  that  can  be  expressed  as  a  smooth  function  of  estimated  totals  of 
one  or  more  variables  (Krewski  &  Rao,  1981). 

Through  the  creation  of  student  replicate  vreights  (defined  below),  the 
jackknife  procedure  allows  the  measurement  of  variability  attributable  to  the 
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use  of  poststratification  and  other  weight  adjustmer/c  factors  that  are 
dependent  upon  the  observed  sample  data.    Once  these  replicate  weights  are 
derived,  it  is  a  straightforward  matter  to  obtain  the  jackknife  variarce 
estimate  of  any  statistic. 

The  jackknife  procedure  in  this  application  involves  first  defining 
pairs  of  first-stage  sampling  units  or  of  appropriate  aggregates  of  them  in  a 
manner  that  models  the  design  as  one  in  which  two  first-stage  units  are  drawn 
with  replacement  per  stratum  in  a  manner  reflective  of  the  actual  design. 
(The  term  first-stage  sampling  unit  is  used  here  in  an  effort  to  avoid  the 
confusion  that  may  urise  because  it  has  been  convenient  and  traditional  to 
retain  the  term  PSU  for  large  geographic  sample  units,  r^any  of  which  are 
included  in  the  sample  with  certainty.)    This  requires  different  approaches 
for  the  certainty  and  for  the  noncertainty  strata.    For  the  main  sample,  the 
60  noncertainty  PSUs  were  formed  into  30  pairs  of  PSUs,  where  the  pairs  were 
composed  of  PSUs  from  adjacent  strata  within  each  subuniverse  (thus  the  strata 
were  relatively  similar  on  the  characteristics  of  proportion  minority 
population,  population  change  between  1970  and  1980,  and  the  proportions  of 
urban  and  farm  populations).    Whereas  the  actual  sample  design  was  to  select 
one  PSU  with  probability  proportional  to  size  from  each  of  60  strata,  for 
variance  estimation  purposes,  the  design  is  regarded  as  calling  for  the 
selection  of  two  PSUs  with  probability  proportional  to  size  with  replacement 
from  each  of  30  strata.    This  procedure  likely  gives  a  small  positive  bias  to 
estimates  of  sampling  error.    The  certainty  PSUs  are  treated  as  strata  for 
variance  estimation  purposes  (which  they  in  fact  are),  with  the  largest 
certainty  PSU  being  treated  as  two  strata  (this  is  appropriate  since 
stratified  selection  was  used  within  PSUs),  and  all  the  other  33  PSUs  being 
treated  as  a  single  stratum  each.    The  schools  in  each  of  the  35  strata  so 
defined  were  the  first-stage  sampling  units,  and  were  assigned  within  each 
strata  to  one  of  two  half -samples ,  with  equal  probability,  and  systematically. 
For  those  certainty  PSUs  with  both  winter  and  spring  assessed  schools,  each 
half -sample  was  composed  of  half  winter  schools  and  half  spring  schools,  so  as 
not  to  reflect  inappropriately  between  season  differences  as  a  component  of 
sampling  error.    For  the  noncertainty  pairs,  one  PSU  was  assigned  at  random  to 
each  half -sample. 

This  procedure  thus  gives  65  half -sample  pairs:    30  from  noncertainty 
PSUs  and  35  from  certainty  PSUs.    About  one- third  of  the  U.S.  population  lives 
within  the  35  certainty  PSUs  and  thus  the  total  number  of  half- sample  pairs 
from  this  sector  can  be  reduced  somewhat,  without  appreciably  reducing  the 
effective  degrees  of  freedom  available  for  variance  estimation  (see  section 
8.4) •    Thus  each  of  the  26  pairs  formed  by  the  26  smallest  certainty  PSUs  was 
combined  with  another  pair  from  the  same  group.    Thus  these  PSUs  constitute  13 
half-sample  pairs*    For  each  such  pair,  a  given  half-sample  contains  half  of 
the  schools  (and  hence  their  students)  from  each  of  two  PSUs.     The  end  result 
was  52  jackknife  pairo  of  half -samples — two  containing  students  from  the 
largest  certainty  PSU,  seven  containing  students  each  from  a  single  large 
certainty  PSU,  13  containing  students  each  from  two  smaller  certainty  PSUs, 
with  the  schools  from  each  PSU  split  between  the  half -samples,  and  30 
containing  students  each  from  two  noncertainty  PSUs,  one  PSU  per  half- sample 
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Similar  pairings  and  half-samples  were  defined  for  the  bridge 
assessments  and  fc»r  Che  season- specific  administrations  of  the  main 
assessment,  but  t:c-*Ki  correspond  to  somewhat  different  clusters  of  PSUs  and, 
since  fewer  PSUs  >are  involved,  smaller  numbers  of  pairs  were  defined  in  each 
case.     Further  iuc^rraation  on  the  construction  of  jackknife  pairs  can  be  found 
in  Rust  et  al.(r;50). 

NAEP's  jack:cnife  variance  estimator  is  designed  for  the  situation  where 
the  first-stage  units,  or  appropriate  aggregates  of  them,  are  paired  within 
strata •    It  esr.xn^ates  the  sampling  variability  of  any  statistic  as  the  sum  of 
components  of  variability  that  may  be  attributed  to  each  of  the  jackknife 
pairs.    The  variance  attributed  to  a  particular  jackknife  pair  is  measured  by 
estimating  how  much  the  value  of  the  statistic  would  change  if  the  information 
embodied  in  the  jackknife  pair  were  to  be  changed.    This  is  done  by  the 
computation  of  a  quantity  t^^  called  a  pseudoreplicate,  which  is  associated 
with  the  i^  jackknife  pair,  and  which  is  an  estimate  of  the  statistic  of 
interest  t  based  on  an  altered  sample.    Specifically,  the  i^^  pseudoreplicate 
of  the  statistic  t  is  created  by  randomly  designating  the  half- sample  membt^rs 
of  the  pair  as  first  and  second,  eliminating  the  data  from  the  first  half- 
sample  of  the  pair,  replacing  the  lost  information  with  that  from  the  second 
half-sample  of  the  pair  (so  chat  the  second  half- sample  is  included  twice), 
repoststratlfying  the  weights,  and  then  reestimating  the  statistic  for  the 
pseudo replicates  based  on  this  altered  set  of  data. 

The  component  of  the  sampling  variability  attributable  to  a  jackknife 
pair  is  estimated  as  the  squared  difference  between  the  value  of  the  statistic 
for  the  complete  sample  and  the  pseudoreplicate  associated  with  the  pair.  The 
estimated  sample  variance  of  the  statistic  t  is  the  sum  of  squared 
difference:?  (where  M  is  the  number  of  jackknife  pairs  define!): 

M 

Var(t)  -         2      (t,  -  t)2 
i-1 

The  statistic  for  the  pseudoreplicate  associated  with  a  given  jackknife 
pair  is  the  original  statistic  for  the  pseudoreplicate  recomputed  using  an 
altered  set  of  weights,  referred  to  as  the  student  replicate  weights.  The 
student  replicate  weight,  SROTi,  for  the  i^  pair  of  first-stage  units  is 
computea  as  follows: 

1)  Let  Wg  be  the  nonrespcnse  adjusted  base  weight  of  a  student,  where 
Wg  accounts  for  the  probabilities  of  selection  and  nonresponse  buc 
does  not  include  poststratif ication  adjustments. 
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2)       Let  Wbi  be  the  nonresponse  adjusted  replicate  base  weight  formed 

by  replacing  the  second  member  of  the  jackknlfe  pair  by  the  first, 
specifically: 


0  If  the  student  Is  In  the  first  set  of 

first-stage  units  In  jackknlfe  pair  1 

JF  *  Wg         If  the  student  Is  In  the  second  set  of 

first-stage  units  In  jackknlfo,  pair  1 

Wg  If  the  student  Is  In  neither  of  the 

first- stage  units  In  jackknlfe  pair  1 


where  JF  Is  a  constant  multiplier  (usually  equal  to  2)  designed  to 
maintain  certain  population  totals. 

3)        Then  the  student  replicate  weight  for  the  jackknlfe  pair  1  Is 
obtained  by  applying  the  poststratlf Icatlon  adjustments  to  the 
weights  Wgj^  In  the  associated  pseudorepllcate . 

The  poststratlf Icatlon  adjustments  are  i  computed  for  each  jackknlfe 
replicate  to  reflect  more  completely  the  total  effect  of  replacing  one  member 
of  a  jackknlfe  pair  with  the  other.     (Nonrespcuse  adjustments  are  not 
recomputed  since  these  are  generally  performed  within  the  PSU  level  and 
therefore  their  effect  is  appropriately  reflected  in  the  variance  estimate.) 

This  estimation  technique  was  used  by  NAEP  to  estimate  all  sampling 
errors  presented  in  the  various  reports.    A  similar  procedure  was  followed  to 
estimate  the  sampling  variability  for  statistics  based  on  any  of  the  bridge 
samples.    The  only  difference  was  in  the  number  of  jackknlfe  pairs  (and  hence 
replicate  weights)  used. 

A  further  discussion  of  the  variance  estimation  procedure  used  by  NAEP 
including  a  discussion  of  alternative  jackknlfe  estimators  that  were  also 
considered  appears  in  John  on  (1989). 

We  noted  above  (as  discussed  in  Chapter  9)  that  a  separate  estf  iiate  o^" 
the  contribution  to  variance  due  to  the  imprecision  in  the  measure  of 
Individual  prof iciwXicles  is  made  and  added  to  the  jackknlfe  estimate  cf 
variance.    That  variance  component  could  have  been  appropriately  reflected  in 
the  jackknlfe  variance  estimates  simply  by  separately  applying  the  XRT 
computations  to  each  jackknlfe  replicate.    Because  of  the  heavier  ''RT 
computational  load,  this  was  not  done.    Less  work  was  Involved  by  the  simple 
procedure  of  making  separate  estimates  of  this  component  to  be  added  to  the 
jackknlfe  variance  estimates.    Also,  a  separate  measure  of  this  component  of 
variance  is  then  available,  which  would  not  be  so  if  it  were  reflected  in  the 
jackknlfe  variance  estimate. 
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8.3      APPROXIMATING  THE  SAMPLING  VARIANCE  USING  DESIGN  EFFECTS 


In  practical  terras,  the  major  expenditure  of  resources  in  the 
computation  of  a  jackknife  variance  estimate  occurs  in  the  preparation  of 
estimates  for  each  of  the  pseudoreplicates .     In  the  1988  assessment,  this 
implies  .r:hat  the  statistic  of  interest  h::s  to  be  recomputed  up  to  53  times, 
once  for  the  overall  estimate  t,  and  once  for  each  of  the  up  to  52 
pseudoreplicates  t^.     Because  this  is  a  considerable  increase  in  the  amount  of 
computation  required,  relative  to  a  conventional  variance  estimate,  it  is  of 
interest  to  see  how  much  the  jackknife  variance  estimates  differ  from  their 
less  computationally  intensive,  simple  random  sampling  based,  analogues.  For 
this  purpose,  we  will  compare  the  jackknife  variance  estimates  with  the 
conventional  estimates  for  three  commonly  computed  statistics:  item 
proportion-correct  statistics,  mean  proficiency  estimates  for  subgroups,  and 
simple  regression  coefficients. 

The  comparison  of  the  conventional  and  the  jackknife  r**.     ods  of  variance 
estimation  will  be  in  terms  of  a  statistic  called  the  design  t^ffect,  which  was 
developed  by  Kish  (1965)  and  extend.^.d  by  Kish  and  Frankel  (1974).    The  design 
effect  for  a  statistic  is  the  ratio  of  the  actual  variance  of  the  statistic 
(taking  the  sample  design  into  account)  over  the  conventional  variance 
'jstimate  based  on  a  simple  random  sample  with  the  same  number  of  elements. 
The  design  effect  is  the  inflation  facto*:  to  be  applied  to  the  conventional 
variance  estimate  in  order  to  adjust  e.  ror  estimates  based  on  simple  random 
sampling  assumptions  to  account  approximately  for  the  effect  of  the  sample 
design.    The  value  of  the  design  effect  depends  on  the  type  of  statistic 
computed  and  the  variables  considered  in  a  particular  analysis  as  well  as  the 
combined  clustering,  stratification,  and  weighting  effects  occurring  among 
sampled  elementc.    Generally,  the  design  effects  for  statistics  from  complex 
samples  such  as  NAEP  are  greater  than  one,  because  variances  based  on  simple 
random  s dimpling  assumptions  tend  to  provide  underestimates. 

8.3.1  Design  Effects  for  Proportion-correct  Statistics 

As  an  example  of  the  distribution  of  design  effect's  to  be  expected  from 
NAEP  data,  we  first  consider  the  design  effect  for  the  statistic  P,  the 
estimated  proportion  of  a  specified  subgroup  of  the  population  who  would 
correctly  respond  to  a  given  assessment  item.    The  proportion-correct 
statistic  is  the  weighted  mean  of  the  responses  to  the  item  of  the  assessed 
individuals  who  belong  to  the  subgroup,  where  an  individual's  response  is 
either  1-correct  or  0-incorrect.    The  design  effect  for  the  proportion- correct 
statistic  P  is  of  the  form 

deff(P)  -  [VarjK(P)]/[P(l  -  P)/N]  . 

In  the  above,  N  is  the  total  number  of  individuals  in  the  subgroup  rersponding 
to  the  item,  VarjK(P)  is  the  jackknife  variance  of  P,  and  P(l  -  P)/N  is  the 
conventional  variance  estimate  of  P.     (Although  the  estimate  P(l  -  P)/N  has 
the  same  form  as  the  simple  random  sampling  estimator  of  the  variance  of  a 
proportion  correct,  the  use  cf  sample  weights  in  the  estimation  of  P  reflects 
the  appropriate  distribution  of  the  population.) 
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The  distributions  of  estimated  design  effects  across  items  for 
oportions  correct  by  grade  and  by  demographic  subgroup  within  grade  across 
all  cognitive  reading  items  presented  in  the  1988  main  assessment  of  reading 
are  indicated  in  Tables  8-6  through  8-8. 

Table  8-6  aadresses  the  distributions  of  the  design  effects  for  the  81 
multiple -oho ice  cognitive  reading  items  presented  in  1988  to  grade  4  students. 
These  distributions  are  sh^vm  for  the  population  as  a  whole  (Total)  as  well  as 
for  a  variety  of  demographic  subgroups:    gender;  race/ethnicity  (White,  Black, 
Hispanic,  other);  age  (less  than  mortal  age,  modal  age,  greater  than  modal 
age);  region  (Northeast,  Southeast,  Central,  West);     size  and  type  of 
community  (rural,  low  metropolitan,  high  metropolitan,  big  city,  urban  fringe, 
medium  city,  small  place);  parental  education  (at  most  high  school,  graduated 
high  school,  post-high  school,  graduated  college,  unknown);  and  type  of  school 
(public,  nonpublic).    For  each  of  these  groupings  of  fourth-grade  students. 
Table  8-6  provides  the  lower  quartile  (LoQ) ,  median,  upper  quartile  (HiQ)  and 
maximum  design  effect  as  well  as  the  me&n  design  effect  for  the  81  multiple- 
choice  items  presented  to  fourth- grade  students. 

Equivalent  information  on  the  distributions  of  design  effects  for  the  99 
multiple -choice  cognitive  reading  items  presented  to  grade  8  students  appears 
as  Table  8-7.    The  108  mu' tiple-choice  cognitive  reading  items  presented  to 
grade  11  students  are  addressed  by  Table  8-8. 

The  particular  demographic  variables  shown  (gender,  race/ethnicity,  age, 
region,  parental  education,  and  size  and  type  of  community)  were  selected 
because  (1)  they  are  major  variables  in  NAEP  reports  and  (2>  they  reflect 
different  types  of  divisions  of  the  population  that  might  have  different 
levels  of  sampling  variability. 

The  tables  show  that  the  design  effects  are  predominantly  larger  than  1, 
indicating  that  standard  variance  estimation  formulas  will  be  generally  too 
small,  sometimes  markedly  so.    Although  the  distributions  of  design  effects 
appear  somewhat  different  fr.  certain  subgroups  of  the  population,  they  are, 
perhaps,  similar  enough  (at  least  within  a  grade)  to  select  an  overall 
composi^te  value  that  is  adequate  for  most  purposes,     in  choosing  a  composite 
design  effect,  some  consideration  must  be  made  about  the  relative  consequences 
of  overestimating  the  variance  as  opposed  to  underestimating  the  variance 
For  example,  adopting  the  position  that  an  overestimate  of  the  variance  is  as 
severe  an  error  as  an  underestimate  leads  to  using  a  composite  that  is  near  to 
the  center  of  the  distributions  of  the  design  effects.     Possible  composites  of 
this  type  are  the  mean  and  median  design*  effects  across  the  combined 
distribution  of  all  design  -ffects.     In  the  current  data,  the  mean  design 
effects  are  1.36,  1.31,  and  1.36  respectively  for  grades  4,  8,  and  12.  These 
are  close  to,  but  greater  than,  the  median  design  effects:     1.29,  1.26,  1.30. 

Alternatively,  one  can  adopt  the  position  that  it  is  a  graver  error  to 
underestimate  the  variability  of  a  statistic  than  to  overestimate  it.  For 
example,  Johnson  and  King  (1987)  examine  estimation  of  variances  using  design 
effects  (among  other  techniques)  under  the  assumption  that  the  consequences  of 
an  underestimate  are  three  times  as  severe  as  these  of  an  overestimate  of  tne 
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Table  8-6 

Distributi     $  of  Design  Effects  Across  Items 
by  Demographic  Subgroup 
for  Proportion-correct  Statistics 
for  the  Cognitive  Reading  Items  Given  in  1988 

Grade  4* 


LoO 

HIO 

Hax 

Mean 

TOT  AT 

1  90 

1  39 

1.60 

2 . 35 

1.44 

MAT  P 
nt\i  «r« 

lis 

1.52 

2.  50 

1.  38 

1  7L 

1.45 

2.44 

1.27 

X  •  £.\J 

1.  54 

2.08 

1.29 

X  ■ 

1  18 

X  •  xu 

1.40 

2.  50 

1.25 

1  11 

1 . 37 

1.59 

2.  32 

1.38 

1.04 

1.17 

1.41 

1.96 

1.19 

<  MODAT. 

0.76 

1.07 

1.29 

2.12 

1.03 

AT  MODAL 

1.00 

1.  31 

1.48 

1.96 

1.27 

>  MODAL 

1.17 

1.35 

1.61 

2.27 

1.41 

NE 

1.14 

1.41 

1.89 

2.68 

1.52 

SE 

1.13 

1.52 

1.94 

3.48 

1.63 

CENTRAL 

0.S9 

1.21 

1.67 

3.00 

1.31 

WEST 

0.96 

1.29 

1.69 

3.93 

1.40 

RURAL 

0.69 

0.99 

1.30 

3.08 

1.09 

LOW  MET 

1.13 

1.52 

1.88 

4.06 

1.59 

HI  MET 

0.96 

1.19 

1.51 

2.86 

1.24 

BIG  CITY 

1.51 

1.96 

2.43 

3.43 

2.03 

FRINGE 

0.99 

1.24 

1.82 

3.30 

1.41 

MED  CITY 

1.36 

1.74 

2.09 

3.22 

1.76 

SMALL  PL 

1.07 

1.34 

1.63 

2.15 

1.37 

<  HS 

0.95 

1.19 

1.43 

2.26 

1.21 

GRAD  HS 

i.Ol 

1.20 

1.36 

1.70 

1.18 

HS  + 

0.99 

1.14 

1.43 

1.84 

1.19 

GRADCOL 

1.02 

1.16 

1.42 

1.90 

1.22 

IDK 

1.11 

1.23 

1,44 

2.53 

1.29 

PUBLIC 

1.24 

1.40 

1.68 

2.46 

1.47 

NON-PUB 

l.C 

1.24 

1.53 

2.25 

1.32 

*  Distributions  are  based  on  81  multiple-choice  items. 
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Table  8-7 


Distributions  of  Design  Effects  Across  Items 
by  Demographic  Subgroup 
for  Proportion-correct  Statistics 
for  the  Cognitive  Reading  Items  Given  in  1988 

Grade  8* 


Group 

LoO 

Median 

HiO 

Ha?c 

Mean 

TOTAL 

1.17 

1.36 

1.57 

2.24 

1.38 

MALE 

1. 11 

1.30 

1.57 

2.55 

1.36 

FEMALE 

1.06 

1.23 

1.36 

1.94 

1.24 

WHITE 

1.18 

1.38 

1.55 

2.57 

1.39 

BLACK 

0.98 

1.25 

1.50 

2.46 

1.28 

HISPANIC 

1.05 

1.26 

1.56 

2.57 

1.32 

OTHER 

0.97 

1.17 

1.35 

2.50 

1.20 

<  MODAL 

0.69 

1.03 

1.29 

2.38 

1.01 

AT  MODAL 

1.11 

1.30 

1.52 

1.95 

1.32 

>  MODAL 

1.02 

1.21 

1.37 

1.94 

1.21 

HE 

0.96 

1.22 

1.52 

2.89 

1.30 

SE 

0.92 

1.32 

1.63 

2.85 

1.35 

CENTRAL 

1.16 

1.61 

2.07 

3.61 

1.67 

WEST 

0.93 

1.25 

1.69 

3.70 

1.40 

RURAL 

0.84 

1.18 

1.67 

4.15 

1.32 

LOW  MET 

0.88 

1.15 

1.50 

2.95 

1.22 

HI  MET 

0.98 

1.38 

1.72 

2.98 

1.41 

BIG  CITY 

1.19 

1.51 

1.88 

4.34 

1.59 

FRINGE 

1.06 

1.28 

1.59 

2.37 

1.34 

riED  CITY 

0.94 

1.22 

1.58 

3.14 

1.30 

SMALL  PL 

1.03 

1.32 

1.65 

2.66 

1.34 

<  HS 

1.10 

1.26 

1.42 

2.19 

1.26 

GRAD  HS 

1.03 

1.24 

1.45 

1.82 

1.24 

HS  + 

1.00 

1.17 

l.liO 

2.06 

1.21 

GRADCOL 

1.01 

1.20 

1.47 

2.30 

1.25 

TDK 

1.07 

1.21 

1.39 

1.99 

1.23 

PUBLIC 

1.13 

1.35 

1.58 

2.29 

1.38 

NON-PUB 

0.95 

1.24 

1.48 

2.32 

1.24 

*  Distributions  are  based  on  99  multiple -choice  items. 
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Table  8-8 


Distributions  of  Design  Effects  Across  Iteras 
by  Demographic  Subgroup 
for  Proportion-correct  Statistics 
for  the  Cognitive  Reading  Items  Given  in  1988 

Grade  12* 


Group 

LoQ 

Median 

HID 

Mean 

TOTAL 

1.21 

1.42 

1.60 

2.76 

1.42 

MALE 

1.16 

1.36 

1.56 

2.34 

1. 

FEMALE 

1.13 

1.32 

1.53 

2.29 

1.37 

WHITE 

1.11 

1.35 

1.49 

2.31 

1.32 

BIACK 

1  .15 

1.38 

1.63 

2.66 

1.42 

HISPANIC 

1.18 

1.39 

1.62 

3.72 

1.44 

OTHER 

1.04 

1.21 

1.49 

2.03 

1.28 

<  MODAL 

0.70 

1.01 

1.33 

2.27 

1.03 

AT  MODAL 

1.19 

1.41 

1.66 

2.67 

1.41 

>  MODAL 

1.04 

1.23 

1.40 

2.26 

1.25 

NE 

1.06 

1.26 

1.68 

3.20 

1.39 

SE 

0.87 

1.10 

1.57 

3.29 

1.23 

CENTRAL 

0.79 

1.13 

1.49 

4.07 

1.23 

WEST 

1.06 

1.31 

1.72 

3.61 

1.42 

RURAL 

0.70 

1.05 

1.57 

3.51 

1.18 

LOW  MET 

1.02 

1.38 

1.69 

5.65 

1.45 

HI  MET 

1.00 

1.39 

1.85 

3.82 

1.49 

BIG  CITY 

1.06 

1.31 

1.75 

3.73 

1.49 

FRINGE 

0.73 

1.07 

1.43 

2.26 

1.13 

MED  CITY 

0.94 

1.24 

1.90 

3.60 

1.41 

SMALL  PL 

1.08 

1.42 

1.81 

2.96 

1.44 

<  HS 

1.11 

1.30 

1.57 

2.73 

1..''5 

GRAD  HS 

1.10 

1.26 

1.45 

2.17 

1.30 

HS  + 

1.06 

1.23 

1.39 

2.17 

1.25 

GRADCOL 

1.12 

1.32 

1.55 

2.12 

1.35 

IDK 

1.13 

1.41 

1.80 

4.83 

1.57 

PUBLIC 

1.21 

l.Al 

1.58 

2.33 

1.41 

NON-PUB 

1.09 

1.45 

1.84 

2.75 

1.50 

*  Distributions  are  ba'sed  on  108  multiple-choice  items. 
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same  magnitude.    Adopting  a  loss  function  that  is  a  weighted  sum  of  absolute 
values  of  the  deviations  of  predicted  from  actual,  with  underestimates 
receiving  three  times  the  weight  of  overestimates,  produces  the  upper  quartile 
of  the  design  effects  as  the  composite  value.    This  assumes  that  the 
distribution  of  design  effects  is  roughly  independent  of  the  jackknife 
estimates  of  variance,  so  that  the  size  of  a  design  effect  does  not  depend  on 
the  size  of  the  variance.    The  values  of  this  composite,  for  grades  4,  8,  and 
12,  respectively,  are  1.59,  1.54,  and  1.59. 

8*3,2  Design  Effects  for  Subgroup  Mean  Proficiency  Scores 

Since  most  of  the  analyses  conducted  by  NAEP  are  based  on  the  results  of 
scaling  models  that  summarize  performance  of  students  across  a  learning  area, 
we  next  consider  the  design  effects  to  be  expected  for  analyses  based  on  these 
scalr  scores.    For  reasons  given  in  Chapter  9,  NAEP  provides  each  individual 
with  a  set  of  "plausible  values,"  each  of  which  is  a  random  draw  from  the 
distribution  of  the  potential  scale  scores  for  that  individual.    Since  our 
current  interest  is  on  the  effect  of  the  sampling  design  on  estimation  and 
inference,  we  will  restrict  oui  attention  to  a  single  measure  of  an 
individual's  proficiency,  the  first  plausible  value  of  the  individual's  scale 
score. 

A  key  statis^-lc  of  interest  is  the  estimated  mean  proficiency  of  a 
subgroup  of  the  po^^ulation.    An  estimate  of  the  subgroup  mean  proficiency  is 
the  weighted  mean  of  the  first  plausible  values  of  prof iciency^of  the  sampled 
individuals  who  belong  to  the  subpopulation  of  interest.    Let  Y  be  the 
weighted  mean  of  the  plausible  values  of  the  sampled  members  of  the 
subpopulation.    The  conventional  estimate  of  the  variance  of  Y  is 

N 

Vareon(Y)  -    [  2  w^-Cy^  -  Y)^]  /  [N-Wj 
i-1 

where  N  is  the  total  number  of  sampled  individuals  in  the  subpopulation  for 
which  proficiency  scores  are  available,  w^  is  the  weight  of  the  i*^^ 
individual,  y^  is  a  plausible  value  from  the  distribution  of  potential  scale 
scores  for  that  individual,  and  W+  is  the  sum  of  the  weights  across  the  N 
individuals . 

The  design  effect  for  the  subgroup  mean  proficiency  estimate  is 
deff(Y)  -  VarjK(Y)  /  Var.^nCY) 

where  Varj,^(Y)  is  the  jackknife  variance  of  Y.     (As  has  been  pointed  out 
previously,  Varj^CY)  as  computed  does  not  measure  the  variability  of  Y  du^  co 
imprecision  in  the  measurement  of  the  proficiencies  of  the  sampled 
individuals.    The  estimation  of  this  very  important  source  of  variability  is 
discussed  in  Chapter  9.) 

Values  of  the  average  design  effects  for  subgroup  mean  proficiencies  are 
displayed,  by      ade,  in  Table  8-9.    The  subpopulations  considered  in  this 
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Table  8-9 


Average  Design  Effects 
by  Demographic  Subgroup  and  Grade 
for  Mean  Reading  Proficiency  Scores* 


Group 

Grade  4 

TOTAL 

3.08 

MALE 

2.46 

FEMALE 

2.02 

WHITE 

2.88 

BLACK 

1.57 

HISPANIC 

1.41 

OTHER 

1.23 

<  MODAL 

0.92 

AT  MODAL 

2.23 

>  MODAL 

2.43 

NE 

4.40 

SE 

3.83 

CENTRAL 

3.51 

WEST 

3.15 

RURAL 

2.14 

LOW  MET 

2.42 

HI  MET 

3.05 

BIG  CITY 

6.32 

FRINGE 

3.58 

MED  CITY 

5.29 

SHALL  PL 

3.90 

<  HS 

1.09 

GRAD  HS 

1.51 

HS  + 

0.93 

GRADCOL 

1.59 

IDK 

1.61 

PUBLIC 

3.41 

NON-FOB 

3.61 

Grade  8  Grade  12 


3.10 

2.45 

2.43 

1.79 

2.30 

2.51 

3.72 

2.51 

2.32 

1.44 

1.39 

1.43 

1.14 

1.03 

0.79 

0.94 

2.31 

2.84 

1.88 

1.32 

2.15 

4.09 

2.48 

1.08 

8.35 

2.15 

1.85 

1.96 

3.34 

1.83 

2.43 

2.57 

3.10 

4.11 

4.20 

2.35 

3.11 

1.85 

2.19 

3.24 

3.69 

3.04 

1.25 

1.31 

2.28 

0.84 

1.20 

1.09 

1.78 

2.10 

0.96 

2.68 

3.08 

2.23 

2.36 

4.16 

*  Design  effects  are  based  on  the  conventional  and  jackknife 
variances  of  subgroup  means  of  the  first  plausible  values  of  reading 
proficiency. 
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table  are  identical  with  those  considered  for  the  design  effects  of 
proportion-correct  stav-istics.    It  is  interesting  to  note  that  the  design 
effects  for  subgroup  mean  proficiency  estimates  are  noticeably  larger  than  the 
design  effects  for  the  proportion-correct  stat^stic.*^.    This  suggests  a  larger 
effect  of  clustering  for  mean  proficiency  scores  (which  are  averages  of 
continuous  variables)  than  for  proper tion-correcc  statistics  (which  are 
averages  of  binary  variables).    Perhaps  more  important,  this  increase  in 
design  effect  may  arise  because  of  the  use  of  BIB  spiraling.    Thus,  for  a 
percent  correct  for  an  item,  the  number  of  students  taking  the  item  within  a 
first-stage  sampling  unit  is  less  than  half  the  number  that  contribute  to  the 
average  proficiency  score  for  a  subject  liVe  mathematics  or  science.  About 
7/3  as  many  students  within  a  first-stage  sampling  unit  are  assessed  on  a 
subject  as  on  an  item,  and  contribute  to  an  average  proficiency  score  for  the 
subject.    Consequently,  the  effective  cluster  size  (and  design  effect)  for  an 
average  proficiency  is  likely  to  be  larger — although  net  as  large  as  it  likely 
would  be  if  all  students  assessed  in  a  subject  ^^.re  given  all  items  for  that 
subject , 

As  was  the  case  for  the  design  effects  of  the  proportion-correct 
statistics,  three  candidates  for  an  overall  composite  design  effect  for 
subgroup  mean  proficiencies  are  the  mean,  the  median,  and  the  upper  quartile 
of  the  distribution  of  design  effects.    The  values  of  t^^ese  potential 
composite  valuss,  by  grade,  are  as  follows: 

Grade  4  Grade  8  Grade  12 


Mean  2.70  2.54  2.18 

Median  2.45  2.32  2.13 

Upper  Quartile  3.56  3.10  2.65 

Each  of  these  values  is  roughly  twice  the  equivalent  value  for  the  design 
effects  of  proportion-correct  statistics. 

We  note  that  the  Var^con)^  as  defined  above  is  an  estimate  of  where 
represen*:^,  the  unit  variance  for  a  simple  random  sample  for  the  population 
of  students  from  which  the  sample  is  also  drawn.    This  is  an  appropriate 
estimate  of  the  increase  in  v^.riance  over  simple  random  sampling  from  that 
population.    However,  the  computer  packages  used  for  estimating  the  variance 
may  not  reflect  the  weights  in  estimating  the  unit  variance,  as  given  above, 
but  i.istead  may  provide  an  estimate  of  a  unit  variance  of  the  form 

K 

2  (yi  -  y)VN^ 

In  this  case,  the  unweighted  estimate  of  unit  variance  would  be  appropriate 
for  the  denominator  of  a  design  effect  measure  of  the  increase  in  variance 
over  the  unit  variance  as  estimates  by  the  computer  package.     If  there  is  no 
correlation  between  the  w^  and  y^,  there  would  be  little  difference  between 
the  two. 
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Design  Effects  for  Simple  Regre>ision  Coefficients 


Table  8-10  shows  the  design  effects  for  simple  regression  coefficients 
from  the  weighted  regression  of  the  first  plausible  value  of  reading 
proficiency  on  each  of  the  individual  conditioning  variables.     (The  codings  of 
the  conditioning  variables  appear  in  Appendix  The  conventional  standard 

errors  were  obtained  from  a  standard  weighted  ' agression  in  which  the  weights 
were  scaled  to  add  to  the  total  sample  size,  thereby  more  closel>  representing 
the  true  amount  of  information  available  in  the  construction  of  the  standard 
errors  relative  to  the  standard  errors  that  would  have  been  produced  if  the 
weights  were  left  in  their  original  metric.      Vie  see  from  Table  S-10  that  this 
scaling  of  the  weights  does  not  go  nearly  far  enough  in  terms  of  appropriately 
representing  the  true  sampling  variability  of  the  regression  estimates. 

The  mean,  median,  and  upper  quartile  of  the  distribution  of  the  design 
effects  for  regression  coefficients  are,  by  grade: 

Grade  4  Grade  8  Grade  12 

Mean  2.40  2.02  1.91 

Median  1.78  1.64  1.59 

Upper  Quartile  3.31  2.56  2.14 

The  mean,  median,  and  upper  quar tiles  of  the  distribution  of  design 
effects  for  regression  coefficients  are  distinctly  larger  than  those  of  the 
proportion-correct  statistics,  but  are  somewhat  smaller  than  those  of  the 
subgroup  mean  proficiencies.    This  results  accords  with  the  conjecture  made  by 
Kish  and  Frankel  (1974)  that  the  design  effects  for  complex  statistics  (such 
as  regression  coefficients)  tend  to  be  smaller  than  the  corresponding  design 
effects  for  means  of  the  same  variable. 


8.4      THE  DEGREES  OF  FREEDOM  OF  THE  VARIANCE  ESTIMATE 

It  is  important  to  have  an  indication  of  the  number  of  degrees  of 
freedom  to  attribute  to  the  jackknife  variance  estimator  Var(t},    The  de^  ^es 
of  freedom  of  &  variance  estit^ator  provide  information  on  the  stability  of 
that  estimator:    the  higher  the  number  of  degrees  of  freedom,  the  lower  the 
variability  of  the  estimator.     Ir*  practical  terms,  th  j  number  of  degrees  of 
freedom  of  the  variance  estiiiiator  corresponds  to  the  number  of  residual 
degrees  of  fr^.edum  t^at  can  be  assumed  for  inferential  procedures. 

Since  the  jackknife  procedure  estimates  the  sampling  variability  of  the 
statistic  by  assessing  the  effect  of  change  in  the  .sample  at  the  paired  first - 
stage  sampling  unit  (FSSU)  level,  the  number  of  degrees  of  freedom  of  the 
variance  estimator  Var(t)  will  be  at  most  equal  to  M,  the  number  of  FSSU 
pairs.    The  maximum  number  of  degrees  of  freedom  equals  the  number  of 
independent  pieces  of  information  used  to  gererate  the  variance.     In  the  case 
of  data  from  the  main  assessments,  the  pieces  of  information  are  the  52 
squared  differences  (t^  -  t)^  ,  each  supplyins  at  most  one  degree  of  freedom 
(regardless  of  how  many  individuals  were  sampled  within  any  FSSU). 
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Table. 8-10 


Average  Design  Effects  by  Grade 
for  Simple  Regression  Coefficients  Based  on  Reading  Proficiency  Scores* 


voexticiBnu 

uITSQe  •+ 

X.J/ 

1  R7 
X  .  o  / 

1  86 

X  .  j£. 

X  .  Oh> 

1  '^0 

1  OA 
X  . 

1  71 

X  .  /  X 

1  Zt8 

ACT  AM 

0  SI 

1  14 

X  .  x*^ 

nx  rii^itvu 

Oft 

L  19 

09 

9  8*^ 

9  Q6 

1  49 

flPNTRAT 

'^0 

7  81 

/  •  O  X 

2.13 

WctDX 

«^  • 

9  59 

2.12 

n«  D  • 

1  07 
X  •  u  / 

1 

X  .  *4  J 

1.20 

0  71 

U  •  /  X 

0  Q'i 

0  9S 

X  •  ^+0 

1  SI 

X  •  J  X 

1  9*^ 

X  .  7  ^ 

X  • 

1 

S7 

J  ~  *f  X  X  ^  ilo 

n  7s 

1  9fi 
X  . 

1  96 

xxiyid 

1  91 
X  .  zx 

1  S 

1.96 

XV 

1  70 

1  so 

1  09 

T\T  <;0^TAPT7n 
1 V  0^-^Jt\]S.LAJ 

1  Ztl 

X  .  *-r  X 

1  00 

X  •  w 

X^nJ^V?  nXIiUlvXXI 

1  8Q 

X  .  o  > 

1  99 

X  .  7  ^ 

wu  nw  oxvi!«r« 

9  99 

1  69 

2.50 

9  9L 

2. hi 

2.50 

TTMF  HU 

2.05 

1.59 

1.79 

6.41 

3.05 

2.14 

NO  LUNCH 

6.98 

3.42 

3.61 

MINOR TTY  ^CM 
nxi^uisx  X  X  Own 

3.54 

3.33 

2.69 

xiN  X  Cvixxi  X  ZiU  o  on 

4  72 

3.14 

3.76 

1.05 

1.02 

1.74 

1  88 

1.05 

1.72 

NONPIIRT  TP 

3.33 

1.96 

4.15 

HOMK  HPT  P 

1.24 

1.47 

MHT  TTPAR 
iiuXjX  XJt  rUv 

0.84 

1.05 

1.07 

1  91 

2.09 

1.34 

9  IS 

1.57 

1.36 

COM  RFAD 

1.43 

1.9? 

1.2o 

1.66 

1.19 

1.61 

LOW  ABSENT 

1.45 

1.59 

GRADES 

1.45 

1.17 

COL  PREP 

1.51 

VOC-TECH 

1.13 

2YR-C0L 

1.40 

4YR-C0L 

1.13 

HRS  WKED 

1.22 

ADV  ENGLISH 

1.28 

REM  ENGLISH 

2.58 

*  Regressions  are  based  on  the  first  plausible  values  of  reading 
proficiency. 
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The  ntunber  of  degrees  of  freedom  of  the  sample  variance  estimator  can  be 
strictly  less  than  the  number  of  FSSU  pairs.    For  example,  suppose  that  the 
statistic  t  is  a  mean  for  some  subgroup  and  no  members  of  that  subgroup  can 
come  from  either  FSSU  in  the  i^^  FSSU  pair.     (Examples  of  such  a  subgroup  are 
any  PSU-level  partitioning  of  che  population,  such  as  region.)    In  this 
instance,  neither  member  of  the  FSSU  pair  i  directly  contributes  to  the 

jtimate  of  t,  so  that  the  pseudoreplicate  t^  would  nearly  equal  the  statistic 
t.    If  the  replicate  weights  used  to  generate  t^  had  not  received 
poststratif ication  adjustments,  the  resulting  pseudoreplicate  t^  would  be 
identical  to  the  overall  estimate  t  so  that  (t^  -  t)^  -  0.     In  this  case,  such 
a  FSSU  pair  would  impart  no  information  to  the  variability  of  the  statistic  t 
and  thus  contribute  zero  degrees  of  freedom  to  the  variance.    However,  since 
the  replicate  weights  have  received  poststratif ication  adjustments,  the 
component  (t^  -  t)^  is  measuring  the  effect  of  the  poststratif  ication  on  the 
estimate.    While  being  nonzero,  such  a  component  will  tend  to  be  much  smaller 
in  magnitude  than  the  sqi.ared  difference  (tj^  -  t)^  for  any  PSU  pair  k  that 
doss  contribute  to  the  estimate  of  t  (see  Rust,  1985). 

The  squared  difference  (t^  -  t)^  estimates  o^^  say,  the  contribution  to 
the  sampling  variance  of  the  statistic  t  which  can  be  attributed  to  the  i**^ 
FSSU  pair  and  Var(t)  estimates  the  sum  of  the  contributions  across  all  pairs. 

H 

i-1 

If  the        vary  widely,  as  when  a  few  of  the        are  markedly  larger  than  the 
remainder,  as  in  the  above  case,  then  ^'ar(t)  is  predominantly  estimating  the 
sum  of  these  larger  components,  which  dominate  the  remaining  terms.  The 
effective  degrees  of  freedom  of  Var(t)  in  this  case  will  be  nearer  to  the 
number  of  dominant  terms. 

One  way  to  estimate  how  many  degrees  of  freedom  to  attribute  to  the 
jackknife  variance  estimate  of  a  statistic  t  is  to  match  estimates  of  uhe 
first  wwo  moments  of  Var(t)  to  those  of  a  chi- square  random  variable 
(Satterthwaite,  ]%1).     If  the  t^  are  normally  dis  ^ibuted,  the  effective 
number  of  degrees  of  freedom  using  this  approximation  is 

M 

IS      (ti  -  t)2  ]  2 


1-1 


M 

S      (tj  -  t)* 
i«l 

However,  this  approximation  will  overestimate  the  effective  degrees  of 
freedom  if  the  distribution  of  the  differences  t^  -  t  has  positive  kurtosis 
(Cochraii.  1977,  p.  96).    A  more  direct  way  of  assessing  the  effective  degrees 
of  frsedom  of  a  variance  estimate  is  possible  when  a  number  of  independent 
replicates  of  the  estimate  are  available.     In  such  a  situation,  the  ordered 
values  of  the  replicate  estimates  of  Var(t)  are  compared  with  the  expected 
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values  of  the  order  statistics  of  various  chi-fiquare  distributions.    The  aim 
is  to  find  the  chi-square  distribution  thcxt  mjst  closely  matches  the  empirical 
distribution.    If  the  distribution  of  Var(t)  :s  appro\imately  chi-square.  a 
quantile-quantile  plot  (Chambers,  Cleveland,  Kleiner,  &  Tukey,  1983,  section 
6.2)  of  the  ordered  replicate  estimates  versus  the  expected  values  of  the 
order  statistics  from  the  best  fitting  chi-square  distribution  will  be  close 
to  a  straight  line  through  the  origin.    The  degrees  of  freedom  for  that  best 
fitting  chi-square  distribution  are  taken  to  be  the  effective  degrees  of 
freedom  of  Var(t) . 

This  procedure  will  be  followed  to  estimate  the  effective  degrees  of 
freedom  for  the  variance  estimates  of  the  subpopulation  mean  proficiencies  and 
of  the  regression  coefficients.  A  slightly  different  procedure  will  be  used 
for  the  effective  degrees  of  freedom  of  the  variances  of  weighted  proportion- 
correct  statistics. 

Tt  is  possible  to  estimate  the  number  of  degrees  of  freedom  to  attribute 
to  the  jackknife  variance  estimates  of  the  weighted  proportion-correct 
statistics  by  considering  the  distribution  of  design  effects  for  a  given  set 
of  items  in  a  population  or  subpopulations  (such  as  males  or  total)  under  the 
assumptions  that  the  individual  design  effects  are  all  estimating  the  same, 
underlying,  design  effect  D  and  that  the  variance  estimates  of  all  weighted 
proportion-correct  statistics  have  the  same  degrees  of  freedom,  f . 
Specifically,  assume  that  the  jackknife  variance  estimate,  Vj,  of  the  j*"^ 
weighted  proportion-correct  statistic,  Pj,  is  distributed  like  the  random 
variable  {a^  /  f)  Xf^,  where  Xf^  is  a  chi-square  random  variable  with  f 
degrees  of  freedom  and        is  the  expected  value  of  Vj.     Further  assume  that 
the  expected  value  of  the  conventional  variance  estimate  is  o^/T^,  where  D  is 
the  underlying  design  effect.    Thien,  for  a  sufficiently  large  sample  size,  so 
that  the  conventional  variance  estimate  can  be  taken  to  be  ajVO,   the  design 
effect  of  Pj  will  be  approximately  distributed  like  the  constant  (D/f)  times  a 
chi-square  random  variable  with  f  degrees  of  freedom.     If  the  underlying 
design  effect  D  and  the  degrees  of  freedom  f  ^re  the  same  for  all  Pj,  ther  the 
distribution  of  the  estimated  design  effects  ot   ;he  proportions  correct  across 
the  set  of  items  will  be  approximately  distributed  like  a  multiple  times  a 
chi-square  random  variable  with  f  effective  degrees  of  freedom. 

Under  these  assumptions,  the  value  of  the  effective  degrees  of  freedom 
for  the  design  effects  of  a  given  subpopulation  (such  as  males  in  the  fourth 
grade)  is  found  by  comparing  the  ordered  values  of  the  design  effects  for  all 
of  the  proportion-correct  statistics  for  that  group  with  the  expected  values 
of  the  order  statistics  of  a  sample  of  the  same  size  of  chi-square  random 
variables  with  f  degrees  of  freedom.    The  comparison  is  repeated  for  all 
integral  degrees  of  freedom  f  between  1  and  M  (where  M-52  for  the  main 
assessment).     The  value  of  f  that  minimizes  the  residual  mean-square  of  the 
least-squares  line  through  the  origin  predicting  th    ordered  design  effects 
from  the  chi-square  order  statistics  is  the  effective  degrees  of  freedom  for 
the  set  of  design  effects. 

Table  8-11  shows  the  result  of  this  estimation  of  the  effective  degrees 
of  freedom  of  the  design  effects,  and  hence  the  jackknife  variance  estimates, 
for  the  weighted  proportion-correct  statistics.    The  quality  of  the 
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Table  8-11 

Effective  Degrees  of  Freedom  for  the  Design  Effects 
of  the  Proportion-cor  '*:ct  Statistics 


Group 

Grade  4 

Grade  8 

Grade 

TOTAL 

41 

37 

40 

MALE 

36 

32 

35 

FEMALE 

27 

47 

40 

WHITE 

31 

36 

41 

BLACK 

25 

23 

30 

HISPANIC 

27 

23 

30 

OTHER 

38 

26 

33 

<  MODAL 

16 

13 

13 

AT  HODAL 

36 

46 

41 

>  MODAL 

38 

48 

40 

NE 

17 

15 

11 

SE 

12 

11 

9 

CENTRAL 

11 

12 

10 

WEST 

13 

11 

13 

RURAL 

10 

10 

7 

LOW  MET 

12 

15 

13 

HI  MET 

15 

14 

13 

BIG  CITY 

19 

13 

13 

FRINGE 

13 

25 

11 

MED  CITY 

15 

li 

9 

SHALL  PL 

21 

18 

18 

<  J1S 

22 

35 

32 

GRAD  HS 

39 

38 

36 

HS  + 

34 

35 

43 

GRADCOL 

42 

30 

44 

IDK 

34 

43 

14 

PUBLIC 

38 

33 

42 

NON-PUB 

21 

24 

14 

ERIC 
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approximation  to  the  distribution  of  the  desiga  effects  is  quite  good:  In 
every  case,  the  residual  mean  squared  error  from  the  ^ ^ediction  of  the  ordered 
values  of  the  design  effects  from  the  order  statistics  from  a  chi-square 
distribution  with  the  effective  degrees  of  freedom  was  no  more  than  10  percent 
of  the  variance  of  the  actual  values  and  was  within  5  percent  in  all  but  two 
of  the  84  cases* 

The  numbers  in  Table  8-11  show  that  the  effective  degrees  of  freedom  of 
the  jackknlfe  variance  estimates  are  indeed  no  larger  than  the  number  of  FSSU 
pairs,  and  are,  in  fact,  markedly  smaller  in  some  cases.    There  appear  to  be 
two  major  groupings: 

1)  Variance  estimates  for  weighted  proportion  statistics 
corresponding  to  subpopulations  that  can  appear  in  any  PSU  in  the 
sample.    These  subpopulations  include:  total,  the  two  genders,  two 
of  the  four  race/ethnicity  groups  (Wnite  and  Other),  students  at 
or  above  modal  age,  the  five  levels  of  parental  education  (except 
for  grade  12),  and  public  schools.    The  variance  estimates  for 
these  subpopulations  have  the  highest  number  of  effective  degrees 
of  freedom.    The  estimated  range  is  between  26  and  48,  and  the 
average  is  about  38.     (Some  exceptions  are  Black  and  Hispanic 
students,  age  less  than  modal  age  for  grade,  less  than  high-school 
education,  nonpublic  school  students,  and  parent's  education 
unknown.    Note  that  while  these  classes  can  appear  in  any  PSU, 
they  tend  to  vary  widely  among  PSUs;  or  may  also  be  based  on  a 
very  small  sample  of  students.    The  estimated  effective  degrees  of 
freedom  for  these  range  from  13  to  38,  and  average  about  23.) 

2)  Variance  estimates  corresponding  to  subpopulations  that  can  only 
appear  in  certain  parts  of  the  sample;  such  as  the  4  regions  and 
the  7  size  and  type  of  community  designations.    These  tend  to  have 
the  smallest  effective  degrees  of  freedom.    They  range  between  7 
and  15,  and  average  about  13. 

It  is  also  interesting  to  determine  how  many  degrees  of  freedom  to 
attribute  to  the  variance  estimates  for  the  subpopulation  mean  proficiencies 
and  for  the  regression  coefficients.    Tables  8-12  and  8-13  provide  this 
information.     In  constructing  these  tables,  a  replication  procedure  was  used 
rather  than  that  used  for  Table  8-11  since,  instead  of  having  a  large  number 
of  design  effects  (one  for  each  item)  from  which  to  build  a  distribution,  we 
have  only  a  single  jackknife  variance  for  each  of  the  subpopulation  means  and 
regression  coefficients.    In  order  to  produce  a  distribution  of  variances  for 
each  of  the  coefficients,  the  full  sample  of  respondents  was  split  into  seven 
interpenetrating  subsamples  based  on  the  assessment  booklet  taken.  (Recall 
that  the  main  assessment  of  reading  consisted  of  seven  booklets  at  each 
grade/age  that  were  spiraled  throughout  the  sample.)    Thus,  all  students 
receiving  the  first  reading  ass>*.ssment  booklet  were  placed  in  the  first 
subsample;  all  students  receiving  the  second  booklet  were  placed  in  the  seco^id 
subsample;  nnd  so  on.    The  result  is  seven  samples,  each  of  which  incliule  all 
of  the  PSUs  in  the  full  sample  while  including  roughly  1/7  of  the  students 
within  each  of  the  PSUs.    Because  of  the  reduced  student  sample  size,  the 
jackknife  variance  estimate  based  on  one  of  the.;e  subsamples  will  be  somewhat 
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Table  8-12 

Effective  Degree^  of  Freedom  for  nhe  Design  Effects 
for  Mean  Reading  Proficiency  Scores* 


Group 

Grade  4 

Grade  8 

Grade  12 

TOTAL 

46 

41 

49 

MALE 

35 

26 

52 

FEMALE 

21 

52 

47 

IffllTE 

22 

45 

52 

BLACK 

22 

32 

26 

HISPANIC 

52 

14 

26 

OTHER 

16 

31 

23 

<  MODAL 

16 

7 

12 

AT  MODAL 

23 

49 

52 

>  MODAL 

29 

26 

39 

NE 

18 

15 

22 

SE 

16 

27 

10 

CENTRAL 

10 

11 

11 

WEST 

10 

7 

9 

RURAL 

52 

9 

6 

LOW  MET 

12 

12 

21 

HI  MET 

23 

21 

8 

BIG  CITI 

39 

29 

11 

FRIKGE 

15 

31 

15 

MED  CITY 

27 

16 

20 

SHALL  PL 

14 

14 

52 

<  HS 

52 

52 

22 

GSAO  HS 

48 

34 

52 

HS  ■¥ 

35 

28 

33 

GRADCOL 

49 

^2 

52 

IDK 

27 

37 

5 

PUBUC 

22 

50 

35 

NGN- PUB 

51 

52 

46 

*  Design  effects  are  based  on  the  first  plausible  values  of  reading 
proficiency. 
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Table  8-13 


Effective  Degrees  of  Frotdom  for  the  Design  Effects 
for  Simple  Regression  Coefficients  Based  on  Reading  Proficiency  Scores* 


Coefficient 

Grade  4 

Grade  8 

Grade  12 

FEMALE 

33 

52 

32 

BIACK 

28 

52 

31 

HISPANIC 

52 

18 

15 

ASIAN 

7 

20 

6 

HI  METRO 

23 

20 

10 

OTHER  STOC 

52 

50 

16 

SOUTHEAST 

16 

33 

30 

CENTRAL 

12 

13 

17 

WEST 

13 

18 

22 

H.S. 

34 

35 

52 

>H.S. 

20 

47 

35 

GRAD  COLLEGE 

52 

52 

18 

PARED  MISSING 

38 

15 

3 

3 -4  ITEMS 

52 

48 

37 

>4  TTMS 

44 

52 

40 

TV 

13 

40 

23 

TV  SQUARED 

21 

52 

12 

LANG  MINORITY 

28 

52 

52 

NO  HW  GIVEN 

52 

11 

11 

HOMEWORK 

52 

13 

13 

TIME  HW 

40 

28 

37 

X  LUNCH 

13 

21 

52 

NO  LUNCH 

38 

37 

20 

MINORITY  SCH 

13 

28 

9 

INTEGRATED  SCH 

14 

52 

39 

MODAL  AGE 

17 

30 

52 

>  MODAL  AGE 

20 

25 

40 

NONPUBLIC 

24 

30 

52 

HOME  HELP 

52 

52 

26 

MULTIPAR 

29 

31 

43 

MOM  HOME 

25 

21 

15 

MOM  WORK 

52 

29 

36 

SOM  READ 

52 

52 

41 

LOT  READ 

42 

52 

39 

LOW  ABSENT 

24 

52 

GRADES 

52 

26 

COL  PREP 

24 

VOC-TECH 

22 

2YR-C0L 

17 

4YR-C0L 

42 

HRS  WKED 

14 

ADV  ENGLISH 

18 

REM  ENGLISH 

3 

*  Regressions  are  based  on  the  first  plausible  values  of  reading 
proficiency. 
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larger  than  the  estimator  from  ♦'he  full  sample.    Because  the  effects  of 
clustering  have  been  reduced,  the  design  effects  for  these  replicate  variance 
estimates  will  be  reduced.    However,  since  the  same  number  of  jackknife  pairs 
a      involved,  the  replicate  variance  estimates  may  reasonably  have 
a|. proximately  the  same  number  of  degrees  of  freedom  as  did  the  estimate  from 
the  full  sample. 

For  each  of  tne  seven  subsamples,  jackknife  variances,  conventional 
variances,  and  design  effects  were  estimated  for  each  of  the  subpopulation 
means  and  regression  coefficients.     For  each  subpopulation  mean  and  regrei>sion 
coefficient,    he  best  fitting  chi- square  distribution  to  the  distribution  of 
the  seven  design  effect  estimates  was  found,  in  the  same  manner  analogous  as 
that  used  for  the  design  effects  of  the  proportion-correct  statistics.  The 
resulting  effective  degrees  of  freedom  appear  in  Table  8-12  for  the 
subpopulation  means  and  in  Table  8-13  for  the  regression  coefficients. 

We  see  again  that  the  effective  degrees  of  freedom  of  the  variance 
estimates  tends  to  be  smaller  than  the  number  of  FSSU  pairs,  and  that  this  is 
true  for  each  of  the  subpopulation  means  and  regression  coefficients. 
Unfortunately,  the  most  striking  characteristic  of  these  estimates  of 
effective  degrees  of  freedom  is  their  variability;     the  effective  degrees  of 
freedom  of  a  subpopulation  for  one  grade  can  differ  considerably  from  the 
equivalent  estimate  for  a  different  gjrade.     (Ar  extreme  example  is  the  rural 
subpopulation  mean,  which  has  effective  deg^  ^      of  freedom,  for  grades  4,  8 
and  12,  of  52,  9,  and  6.)    This  variability  may  be  reflecting  the  instability 
of  the  degrees  of  freedom  estimates  since  e^h  estimate  is  based  on  only  7 
values.    The  degrees  of  freedom  estimates  t      the  proportion-correct 
statist icc^  are  much  more  stable  since  these  are  based  on  at  least  81  values. 

The  eff^wtJ^'e  degrees  of  freedom  for  the  NAEP  jackknife  variance 
estimates  are  markedly  smaller  than  the  degrees  of  freedom  attributed  to  the 
corresponding  error  estimates  from  conventional  techniques.    This  fact  affects 
inferential  procedures  since  significance  tests  based  ok  the  conventional 
degrees  of  freedom  will  be  too  lib^-ral  (and  confidence  intervals  will  be  toj 
small).     Fortunately,  for  the  usual  significance  levels,  the  effect  of  using 
the  effective  degrees  of  freedom  rather  than  the  conventional  values  is 
generally  moderate:    a  t  statistic  significant  at  the  a-5%  level  assuming 
infinite  degrees  of  freedom  (  ^sentially  the  conventional  estimate)  is 
significant  at  the  a-6%  level  for  20  effective  degrees  of  freedom,  the  a-7% 
level  for  10  effective  degrees  of  freedom,  and  the  a^lOX  level  for  five 
effective  degrees  of  freedom. 

For  practical  purposes,  the  impact  of  the  reduced  degrees  of  freedom  on 
inferential  techniques  can  be  largely  accounted  Tor  by  (1)  using  a  moderate 
number  (say  25)  of  degrees  of  freedom  f ^r  all  inferences  of  items  that  can 
appear  in  all  PSUs,  and  (2)  using  a  smaller  number  (say  10)  for  the  remaining 
items.     Certainly  one  should  be  cautious  about  barely  significant  results  for 
subgroups  that  are  highly  clustered  in  the  population. 
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Chapter  9 
SCALING  PROCEDURES^ 


Robert  J,  Mislavy 
Educational  Testing  Service 


A  key  innovation  in  NAEP  during  the  ETS  tenure  is  scale -score  reporting. 
With  scaling  methods,  the  performance  of  a  sample  of  students  in  a  subject 
area  or  subarea  can  be  summarized  on  a  single  scale  even  when  different 
students  have  been  administered  different  exercises.    Similar  procedure<:  can 
be  used  to  summarize  responses  to  sets  of  related  background  questions.  This 
chapter  presents  an  cverviev  of  the  scaling  methodologies  employed  in  the 
analyses  of  the  1988  NAEP  surveys: 

•  Section  9.1  briefly  discusses  the  perspective  on 
scaling  from  which  the  procedures  were  conceived  and 
applied. 

•  Section  9.2  reviews  the  "plausible  values"  methodology 
used  in  NAEP  scale -score  analyses. 

•  Section  9.3  describes  how  plausible  values  are  used  in 
subsequent  analyses. 

•  Section  9. A  lists  the  scale-score  analyses  carried  out 
on  the  1988  data. 

Details  of  scaling  procedures  for  specific  subject  areas  are  presented  in 
Chapters  10  through  15. 

9-1      SCALING  IN  NAEP 

NAEP  reports  were  originally  envisaged  some  20  years  ago  as  simple  lists 
of  percents  correct  to  individual  survey  items,  in  the  population  as  a  whole 
and  in  subpopulations  of  particular  interest.    It  soon  became  apparent, 
however,  that  major  features  of  the  detailed  results  from  hundreds  of  items 
could  not  be  effectively  communicated  without  some  kind  of  summarization  The 
Education  Commission  of  the  States,  the  NAEP  contractor  at  that  time, 
introduced  averages  of  percents  correct  for  sets  of  items,  which  summarized 
pattern^  of  performance  in  groups  of  related  exercises  and  made  it  possible  to 
compare  more  easily  the  general  performances  of  subpopulations — as  long  as 


^The  contributions  of  Albert  Beaton,  Eugene  Johnson,  David  Freund,  Bruce 
Kaplan,  Jennifer  Nelson,  Kathleen  Sheehan,  Minhwei  Wang,  and  Rebecca  Zwick  to 
this  chapter  are  gratefully  acknowledged. 

?29 

ERLC 


245 


those  subpopulations  had  been  T)reseuted  a  common  subset  of  items.  Because 
this  approach  limits  comparisons  to  percents  correct  on  specific  sets  of 
items,  it  provides  no  simple  way  to  report  trends  over  time  as  the  item  pool 
e\olves.    Moreover,  it  gives  no  information  about  the  distributions  of  skills 
among  students  in  targeted  subpopulations. 

These  limitations  can  be  overcome  by  the  use  of  response  scaling 
methods •    If  several  items  require  .s^ilar  skills,  the  regularities  observed 
in  response  patterns  can  often  be  exploited  to  characterize  both  respondents 
and  itenis  in  terms  of  a  relatively  small  number  of  variables.    When  combined 
through  appropriate  mathematical  formulas,  these  variables  capture  the 
dominant  features  of  the  data.    Using  the  scale,  it  becomes  possible  to  talk 
about  distribi*tions  of  proficiency  in  a  population  or  subpopulation,  and  to 
estimate  the  relationships  between  proficiency  and  background  variables. 

Early  work  on  scaling  is  attributed  to  Thurstone,  but  the  more  recent 
development  of  item  response  theory  (IRT)  has  been  particularly  inlxuential  on 
measurement  practice  (Hambleton,  1989).     IRT  and  a  newly  developed  procedure 
called  the  average  response  method  (ARM) ,  both  of  which  are  reviewed  in 
section  9.2,  are  the  two  scaling  procedu-es  ETS  has  introduced  in  NAEP 
reporting. 

We  hasten  to  point  out  that  any  procedure  of  aggregation,  from  a  simple 
average  to  a  complex  multidimensional  scaling  model,  highlights  certain 
patterns  at  the  expense  of  other  potentially  Interesting  patterns  that  may 
reside  within  the  data.     In  a  very  real  sense,  every  single  item  In  a  NALP 
survey  is  of  interest  in  its  own  right,  and  can  provide  useful  information 
about  what  young  Americans  know  and  can  do.     The  choice  of  an  aggregation 
procedure  must  be  driven  by  a  conception  tT  just  which  patterns  are  salient 
for  a  particular  purpose.    The  procedure  that  is  optimal  for  one  purpose  may 
be  poorly  suited  for  another. 

The  relatively  high  levels  of  aggregation  found  in  ETS/NAEP  reports  such 
as  The  Reading  Report  Card:    Progress  Toward  Excellence  in  Ouz  Schools  (NAEP, 
1985),  for  example,  are  well  suited  to  high-level  discussions  of  trends  and 
policy  implications.     They  average  over,  and  therefore  are  not  keyed  to,  the 
microanalysis  of  performance  at  tV t  level  of  specific  skills,  as  might  be 
desired  by  educational  psychologists;  they  do  not  reveal  popular  student 
misconceptions  or  erroneous  rules,  as  might  be  of  intere-st  to  classroom 
teachers  in  a  subject  area.    For  studying  specific  skills,  ore  might  nrefer 
the  precision  of  a  latent  class  model  for  more  highly  specified  skil.  For 
studying  misconceptions,  detailed  discussions  of  results  for  individual  item.*^ 
might  be  more  appropriate.     By  no  means  do  the  scale -score  methods  we  employ 
as  a  reporting  vehicle  exh^ist  the  potential  of  NAEP  data;  neither  do  they 
prevent  other  researchers  from  carrying  out  alternative  analyses  fiom 
different  perspectives.     Indeed,  NAEP  public-use  data  tapes,  which  contain  the 
original  responses  of  all  sur^'eyed  students,  were  created  expressly  to 
encourage  and  facilitate  such  analyses. 

A  reporting  scale  in  the  1988  NAEP  survey,  then,  simply  summarizes 
performance  on  a  collection  of  educational  tasks  in  much  the  same  way  that  the 
Consumer  Price  Index  (CPI)  summarizes  the  total  cost  of  a  market  basket  of 
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products.    The  two  indices  exhibit  some  of  the  same  useful  features  and 
limitations.    Just  as  the  CPI  composite  represents  average  American  spending 
patterns,  the  items  in  a  NAEP  survey  were  specified  by  independent  consensual 
process  to  tap  a  "market  basket  of  skills."    Just  as  changes  in  the  CPI 
reflect  at  a  g-^ance  changes  in  the  cost  of  goods  in  general,  changes  in  NAEP 
scale-score  distributions  reflect  changes  in  proficiency  as  averaged  over  the 
items  in  the  pool.     But  understai^.ding  just  how  and  why  the  CPI  changes 
requires  deeper  analyses,  into  specific  components  of  th**.  market  basket,  when 
the  CPI  goes  up,  some  of  the  components  will  have  gone  up  by  greater  rates 
than  others,  while  some  may  have  even  dropped  in  price.    The  NAEP  scale 
depends  similarly  on  the  balance  of  items  of  varying  types  and  topics  in  the 
survey,  and  reflects  only  an  average  over  the  varying  patterns  among  them. 
NAEP  first  attempts  to  carry  out  scaling  in  subject  areas  in  which  similar 
patterns  can  be  expected  over  items;  then,  within  each  scaling  area, 
highlights  meaningful  departures  from  general  trends  in  several  ways, 
including  the  following: 

1)  Explicitly  discussing  countertrends  or  comparisons  that  can  be 
identified  with  one  or  a  few  items.    This  is  analogous  to 
reporting  that  the  Consumer  Price  Index  jumped  5  percent,  but 
noting  that  the  increase  was  mainly  due  to  a  change  in  OPEC  oil 
prices. 

2)  Supplementing  scale-score  distributional  results  with  more 
detailed  breakdowns  in  terms  of  percents  correct  for  groups  of 
related  items. 

3)  Carrying  out  scaling  in  separate  subareas  within  the  subject  area, 
in  which  it  is  anticipated  that  trends  or  comparisons  may  differ 
because  of  different  curricular  emphases  over  time  or  across 
schools.    As  was  done  in  the  1986  NAEP  surveys  of  mathematics  and 
science,  these  subscale  results  were  supplemented  by  a  subject 
area  average.    This  is  comparable  to  calculating  price  changes  in 
separate  market  baskets  for  food,  transportation,  energy,  and  so 
on,  and  reporting  these  individually  along  with  the  overa^'l 
average . 

9.2      NAEP  SCALING  METHODOLOGY 

The  paragraphs  that  follow  review  the  scaling  models  employee  in  the 
analyses  of  1988  NAEP  data,  and  the  "plausible  values"  methodology  that  allows 
such  models  to  be  used  with  NAEP's  sparse  item-sampling  design.    The  reader  is 
referred  to  Mislevy  (1988a,  in  press)  for  an  introduction  to  plausible  values 
methods  and  a  comparison  with  standard  psychometric  analyses,  to  Mislevy  and 
Sheehan  (1987)  and  Beaton  and  Johnson  (1987,  1990)  for  additional  information 
on  how  the  models  are  user^  in  NAEP,  and  to  Rubip  (1987)  for  the  theoretical 
underpinnings  of  the  approach. 
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9.2.1    The  Scaling  Models 


Two  types  of  scaling  models  were  used  by  NAEP  in  the  1988  assessment. 
The  three -parameter  logistic  (3PL)  model  from  item  response  theory  (IRT;  e.g., 
Lord,  1980)  was  used  for  the  subject  areas  of  reading,  civics,  U.S.  history, 
geography,  mathematics,  and  science.    The  average  response  method  (ARM,  Beator. 
6c  Johnson,  1987,  1990),  developed  by  NAEP  for  the  1984  assessment,  was  used 
for  the  subject  area  of  writing.     Both  are  "latent  variable"  models, 
quantifying  respondents'  tendencies  to  provide  responses  in  a  given  direction 
(e.g. ,  correct  answers  to  items  in  a  subject  area;  positive  responses  on 
attitude  questJ.ons;  higher  rather  than  lower  ratings  in  written  essays),  as  a 
function  of  a  parameter  that  is  not  directly  observed. 

The  three -parameter  logistic  (3PL)  IRT         1.    The  fundamental  equation 
of  the  3PL  model  is  the  probability  that  a  pe*       whose  proficiency  is 
characterized  by  the  unobservable  Viiriable  d  v/ill  respond  correctly  to  item  j  . 

P(Xj-l|^aj,bj,Cj)  -  Cj  +  (l-cj)/{l+exp[-l  7a^(e-h^)]) 

-  Pj(^).  (9.1) 

where 

Xj        is  the  response  to  item  j  ,  1  if  correct  and  0  if  not; 

aj,      where  aj>0,  is  the  slope  parameter  of  item  j,  characterizing  its 
sensitivity  to  proficiency; 

bj        is  the  threshold  parameter  of  item  j,  characterizing  its 
difficulty;  and 

Cj,      where  0<Cj<l,  is  the  lower  asymptote  parameter  of  item  j, 

reflecting  the  chances  of  a  correct  response  from  students  of  very 
low  proficiency.     In  1988  NAEP  analyses,  c  parameters  were 
estimated  for  multiple-choice  items,  but  were  fixed  at  zero  for 
open-ended  items. 

For  the  purposes  of  reporting  item  parameter  estimates  and  other 
intermediary  estimates,  the  linear  indeterminacy  apparent  in  (9.1)  may  be 
resolved  by  an  arbitrairy  choice  of  the  origin  and  unit  size  in  a  given  scale. 
This  was  done  for  the  reading  scale  in  1984  by  standardizing  the  combined 
grade  4/age  9,  grade  8/age  13,  and  grade  11/age  17  samples.     To  aid 
interpretation,  final  published  results  are  reported  on  scales  that  are 
transformed  linearly  from  the  8  scale  in  ways  related  to  the  O-to-500  reading 
proficiency  scale  developed  in  the  1984  NAEP  assessment  of  reading  (Beaton, 
1987a).    These  transformations  are  described  in  the  corresponding  subject  area 
chapters  in  this  report. 
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Under  the  usual  IRT  assumption  of  local  independence,  the  probability  of 
a  vector  x-(Xi, . .  .  ,x„)  of  responses  to  n  items  is  simply  the  product  of  terms 
based  on  (9.1) : 


n  Xj  1-Xj 

P(x|fl,a,b,c)  -  n  [Pj(fl)]     [l-Pj(fl)]  .  (9.2) 

j 

It  is  typically  also  assumed  that  response  probabilities  are  conditionally 
independent  of  background  variables,  say  y,  given  fl,  or 

^(x|fl,a,b,c,y)  -  P(x|fl,a,b,c)  . 

(Checks  on  the  degree  to  which  tho-se  assumptions  are  met  in  NAEP  data,  and 
ways  that  mean5.ngful  departures  are  handled,  are  described  b^'low.) 

After  X  has  been  observed,   (9.2)  can  be  viewed  as  a  likelihood  function, 
and  provides  a  basis  for  inference  about  d  or  about  item  parameters.     In  NAEP, 
est.lmates  of  item  parameters  were  obtained  with  a  modified  version  of  Mislevy 
and  Bock's  (1982)  BILOG  computer  program,  then  trea^  d  as  known  in  subsequent 
calculations.    Once  items  have  been  calibrated  in  this  manrer,  a  likelihood 
function  for  6  is  induced  by  a  vector  of  responses  to  any  subset  of  calibrated 
items,  thus  allowing    -based  inference  from  matrix  samples. 

Conditional  independence  is  a  mathematical  assumption,  not  a  necessary 
fact  of  nature.     Even  though  the  IRT  models  are  employed  in  NAEP  only  to 
summarize  average  performance,  a  number  of  checks  are  made  to  detect  serious 
violations  of  conditional  dependence,  and,  when  warranted,  remedial  efforts 
are  made  to  mitigate  its  effects  on  inferences.    Tliese  include  the  following: 

1)  Checks  on  relative  item  operating  characteristics  among  distinct 
gender  and  ethnicity  groups  (i.e.,  differential  item  functioning, 
or  DIF,  analyses).     Some  degree  of  relative  differences  are  to  be 
expected,  of  course,  and  modestl>  varying  profiles  among  groups 
will  exist  beyond  the  differences  conveyed  by  their  differing  6 
dir^tributions.    The  intent  at  this  stage  is  to  detect  and 
eliminate  items  that  operate  different? p1 ly  for  identifiable 
reasons  that  are  unrelated  to  the  skills  intended  to  be  measured 
in  the  subject  area. 

2)  When  a  scale  extends  over  age  groups,  evidence  is  sought  of 
different  operating  characteristics  over  ages.    Whan  such  effects 
are  found,  an  item  in  question  is  represented  by  different  item 
parameters  in  different  age  groups. 

3)  When  a  scale  extends  over  time,  e\idence  is  similarly  sought  as  to 
whether  an  item's  relative  operating  characteristics  have  changed 
over  time,  over  and  above  differences  that  can  be  accounted  for  by 
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changes  in  the  overall  9  distribution.     Studies  of  NAEP  reading 
data  (BeatOT-  &  Zwick,  1990)  suggest  these  effects  are  small  in 
adjacent  asa>3ssments  when  assessment  forms  are  held  constant,  but 
they  too  cpw  be  taken  into  account  by  allowing  item  parameters  to 
vary  over  time  (see  Bock,  Muraki,  &  Pfeif fenberger ,  1988).  The 
variation  in  item  parameters  is  relative  to  other  items  in  the 
scale;  general  trends  in  item  parameters  are  equivalent  to  changes 
in  $  distribution. 

Item-level  factor  analyses  have  diminished  in  importance  as  our 
perspective  of  the  role  of  IRT  in  NAEP  has  evolved.    The  assumption  that 
performance  in  a  scaling  area  is  driven  by  a  single  unidimensional  variable  is 
unarguably  incorrect  in  detail.    But  our  use  of  the  model  is  not  theoretical, 
but  data  analytic;  interpret^ wion  of  results  is  not  trait-referenced,  but 
domain-referenced.    Scaling  ^  ceas  are  determined  a  priori  by  considerations  of 
content  and  politics,  as  collections  of  items  for  which  overall  performance  is 
deemed  to  be  of  interest.    The  IRT  summary  is  not  expected  to  capture  all 
meaningful  variation  in  item  response  data,  but  to  reflect  distributions  of 
overall  proficiency — to  summarize  ♦'he  main  patterns  in  item  percents -correct 
in  the  populations  and  subpopulations  of  interest.    Using  a  unidimensional  IRT 
model  when  the  true  model  is  multidimensional  captures  the.oc  overall  patterns 
even  though  it  over-  or  under-estimates  the  covariances  among  responses  to 
items  in  pairs.    For  inferences  based  on  overall  proficiency,  violations  of 
the  model  with  respect  to  dimensionali*,y  are  less  serious  than  violations  in 
the  shapes  of  the  marginal  response  curves — hence  our  greater  attention  to 
routine  checks  of  item-fit  residuals  for  every  item  in  every  calibration  run 
than  to  factor  analytic  results. 

Once  it  is  accepted  that  the  model  cannot  be  strictly  correct,  attention 
focuses  on  violations  that  distort  the  most  important  inferences  that  are  to 
be  drawn.     Estimated  proficiency  distributions  and  item  parameters,  for 
example,  correctly  capture  groups'  overall  performances,  but  may  over-  or 
under -predict  certain  items.    The  first  type  of  information  is  very  important 
in  NAEP;  the  second  is  less  important.    Note  however  thdt  this  imparts  an 
importance  to  the  mix  of  items  that  would  not  be  present  if  the  IRT  model  were 
true,  since  the  balance  of  items  presented  determines  the  nature  of  average 
proficiency.     Item  mixes  are  identical  for  subpopulations  within  a  given 
assessment,  so  overall  proficiency  automatically  has  the  same  meaning  for 
them.     Item  mixas  are  not  necessarily  the  eame  for  different  grade/age  groups 
or  different  time  points,  so  the  more  careful  checks  listed  above  as  2)  and  3) 
are  required  to  maintain  (as  well  as  possible)  meaning  across  these  linkages. 

In  all  NAEP  IRT  analyses,  missing  responses  at  the  end  of  each  block  a 
studen*:  was  presented  were  considered  "not-reached,"  and  treated  as  if  they 
had  not  been  presented  to  the  respondent.    Missing  responses  before  the  last 
observed  response  in  a  block  were  considered  intentional  omissions,  and 
treated  as  fractionally  correct  at  the  value  of  the  reciprocal  of  the  number 
of  response  alternatives.     These  conventions  are  discussed  by  Mislevy  and  Wu 
(1988).    With  regard  to  the  handling  of  not-reached  items,  they  find  that 
ignoring  not-reached  items  introduces  slight  biases  into  item  parameter 
estimation  to  the  degree  that  (1)  not- reached  items  are  present  and  (2)  speed 
is  correlated  with  ability.    With  regard  to  omissions,  they  find  that  the 

234 

ERIC 

250 


method  described  above  provides  consistent  limited- information  likelihood 
estimates  of  item  and  ability  parameters  under  the  assumption  that  respondents 
omit  only  if  they  can  do  no  better  than  responding  randomly. 


The  local  independence  assumption  embodied  in  (9.2)  implies  that  item 
response  probabilities  depend  only  on  $  and  the  specified  item  paramc  ters — not 
on  the  position  of  the  item  in  the  booklet,  on  the  content  of  items  abound  an 
item  of  interest,  or  on  test-administration  timing  conditions.     These  ettects 
are  certainly  present  in  any  application,  however.    The  pra^     :al  question  is 
whether  the  IRT  probabilities  obtained  via  (9.2)  are  "close  enough"  to  be 
robust  with  respect  to  (1)  the  c.  ntext  in  which  the  data  a      jo  be  collected 
and  (2)  the  inferences  that  are  to  be  dra\m.    For  example,  experience  with 
adaptive  testing  has  shown  using  the  same  item  parameters  regardless  of  when 
an  item  is  administered  does  not  materially  bias  estimates  of  the 
proficiencies  of  individual  examinees.    Our  experience  with  the  1986  NAEP 
reading  anomaly,  has  shown,  however,  that  for  measuring  small  changes  over 
time,  changes  In  item  context  and  speededness  conditions  lead  to  unacceptable 
large  random  error  components.    These  can  be  avoided  by  presenting  items  used 
to  measure  change  in  identical  test  forms,  with  identical  timings  and 
administration  conditions.    Thus  we  do  not  maintain  that  the  item  parameter 
estimates  obtained  in  any  particular  booklet  configuration  are  appropriate  for 
other  conceivable  conf * juracions ,  and  the  parameter  estimates  are  context - 
bound.     (Fot  this  reason,  we  prefer  common  population  equating  to  common  item 
equating  whenever  equivalent  random  samples  are  available  for  linking.)  A 
given  assessment  block,  it  will  be  recalled,  can  appear  as  the  first,  second, 
or  third  block  of  a  booklet.    The  appearances  are  balanced,  so  that  any 
differences  in  item  parameters  due  to  block  position  are  averaged  out. 

The  average  response  method  (ARM)  model.     The  basic  equation  of  the  ARM 
is  an  average  of  item  responses: 


Here  a  is  a  vector  of  constants,  specified  so  as  to  provide  a  meaningful 
summary  of  performance.    Weights  of  1/n  for  an  n-item  test,  for  example,  yield 
simply  an  average  score;  weights  given  by  the  k^^  eigenvector  of  the 
covariance  matrix  for  x  yield  the  k*^^  component  score.     If  a  respondent 
responded  to  all  items,  then  an  ARM  score  would  be  directly  calculable  via 
(9.3)  without  error.    Typically,  however,  a  given  NAEP  respondent  receives 
only  a  subset  of  the  iten  >  in  an  ARM  scale,  so  that  his  or  her  ARM  8  is  not 
observed  directly. 

9.2.2    An  Overview  of  Plausible  Values  Methodology 

Item  response  theory  was  developed  in  context  of  measuring  individual 
examinees'  abilicies.     In  that  setting,  each  individual  is  administered  enough 
items  (often  100  or  more)  to^permit  precise  estimation  of  his  or  her  $ ,  as  a 
maximum  likelihood  estimate  $,  for  example.    Because  the  uiicertainty 


$ 
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associated  with  each  6  is  negligible,  the  distribution  of  ^,  or  the  joint 
distribution  of  6  with  other  variables,  can  then  be  approximated  using 
individuals'  0  values  as  if  thev  were  6  values. 

This  approach  breaks  down  in  the  assessment  setting  when,  in  order  to 
provide  broader  content  coverage  in  limited  testing  time,  each  respondent  is 
adralnistered  relatively  few  items  in  a  scaling  area.    The  problem  is  that  the 
uncertainty  associated  with  individual  ^s  is  too  large  to  ignore,  and  the 
features  of  the  9  distribution  can  be  seriously  biased  as  estimates  of  the  6 
distribution.     (The  failure  of  this  approach  was  verified  in  early  analyses  of 
the  1984  NAEP  reading  survey;  see  Wingersky,  Kaplan,  &  Beaton,  1987.) 
"Plausible  values"  were  developed  as  a  way  to  estimate  key  population  features 
consistently,  and  approximate  others  no  worse  than  standard  IRT  procedures 
would.    A  detailed  development  of  plausible  values  methodology  is  given  in 
Mislevy  (1988a,  in  press).    Along  with  theoretical  justifications,  that  paper 
presents  comparisons  with  standard  procedures,  discussions  of  biases  that 
arise  in  some  secondary  analyses,  and  numerical  examples.    The  following 
paragraphs  give  a  brief  overview  of  the  plausible  values  approach,  focusing  on 
its  implementation  in  the  1988  NAEP  analyses. 

Let  X  ^represent  the  responses  of  all  sampled  examinees  to  background  and 
attitude  questions,  along  with  design  variables  such  as  school  membership.  If 
IRT  or  ARM  9  values  were  available  for  all  sampled  examinees,  it  would  be 
possible  to  compute  a  statistic  t(£.,x) — such  as  a  subpopulation  sample  mean,  a 
sample  percentile  point,  or  a  sample  regression  coefficient — to  estimate  a 
corresponding  population  quantity  T.    A  function  U(£.,x) — e.g.,  a  jackknife 
estimate — would  be  used  to  gauge  sampling  uncertainty,  as  the  variance  of  t 
around  T  in  repeated  samples  from  the  population. 

Because  the  3PL  model  and  the  ARM  are  latent  variable  models,  however,  9 
values  are  not  observed  even  for  sampled  students.    To  overcome  this  problem, 
we  follov  Rubin  (1987)  by  thinking  of  I  as  "missing  data"  and  cpproximate 
t(j?.,x)  t>y  its  expectation  givpn  (x.X).   ^Vic  data  that  actually  were  observed, 
as  follows: 

t*(2C.x)  -  E[t(i.2)|x.y.] 

-  /  t:(i,x)  p(iix.X)  dl    •  (9.4) 

It  is  possible  to  approximate  t*  using  random  draws  from  the  conditional 
distributions  p(^|Xi,yi'^  of  each  sampled  student  i.    These  values  are  referred 
to  as  "imputations"  in  the  sampling  literature,  and  "plausible  values"  in 
NAEP.     The  value  of  6  for  any  respondent  that  would  enter  into  the  cc^iputation 
of  t  is  thus  replaced  by  a  rar  '  mly  selected  value  from  the  conditional 
distribution  for  9  given  his  o.  her  responses  to  cognitive  items  (xj  ana 
background  items  (yi).    Rubin  (1987)  proposes  that  this  process  be  carried  oat 
several  times — "multiple  imputations" — so  that  the  uncertainty  associated  with 
imputation  can  be  quantified.    The  average  of  the  results  of,  say,  K  estimate*^ 
of  t,  each  computed  from  a  different  set  of  plausible  values,  is  a  Monte  Ca 
approximation  of  (9.4);  the  variance  among  them,  B,  reflects  uncertainty  due 
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to  not  observing  9,  and  must  be  added  to  the  estimated  expectation  of  U(i,x) » 
which  reflects  uncertainty  due  to  testing  only  a  sample  of  students  from  the 
population.    Section  9.3  explains  how  plausible  values  are  used  in  subsequent 
analyses . 

It  cannot  be  emphasized  too  strongly  that  plausible  values  are  not  test 
scor^^a  for  individuals  in  the  usual  sense. 

Plausible  values  are  offered  only  as  intemediary  computations  for 
calculating  integrals  of  the  form  of  equation  9.4,  in  order  to  estimate 
population  charactaristics.    When  the  underlying  model  is  correctly  specified, 
plausible  values  will  provide  consistent  estimates  of  population 
characteristics,  even  though  they  are  not  generally  unbiased  estimates  of  the 
proficiencies  of  the  individuals  with  whom  they  are  associated.    The  key  idea 
lies  in  a  contrast  between  plausible  values  and  the  more  familiar  6  estimates 
of  educational  measurement  that  are  in  some  sense  optimal  for  each  examinee 
(e.g. ,  maximum  likelihood  estimates,  which  are  consistent  estimates  of  an 
examinee's  fl,  and  Bayes  estimates,  which  provide  minimum  mean-squared  errors 
with  respect  to  a  reference  population):    Point  estimates  that  are  optimal  for 
individual  examinees  have  distributions  that  can  produce  decidedly  noncptimal 
(specifically ,  inconsistent)  estimates  of  population  characteristics  (Little  &. 
Rubin,  1983).     Plausible  values,  on  the  other  hand,  are  constructed  explicitly 
to  provide  consistent  estimates  of  population  effects. 

In  both  IRT  and  ARM  analyses  in  NAEP,  plausible  values  are  included  for 
the  small  numbers  of  students  who  responded  to  background  questions,  but, 
although  they  were  presented  items  in  a  subject  area,  did  not  respond  to  any 
of  them.     The  conditional  distribution  employed  for  such  a  nonrespondent  is 
based  solely  on  his  or  her  background  values  y.    This  special  class  of 
nonrespondents  was  included  in  this  manner,  even  thou£,h  they  provided  no 
information  about  their  proficiencies,  in  order  to  maintain  the 
representativeness  of  the  sample.    This  convention  provides  estimates  of 
population  characteristics  that  have  the  same  expected  value  and  precision  as 
would  be  obtained  under  the  mire  familiar  nonresponse  adjustment  of  deleting, 
the  nonrespondents  and  boosting  the  sampling  weights  of  respondents  with  the 
same  y  values,  since  their  plausible  values  are  drawn  from  the  estimated  6 
distributions  of  the  appropriately  matched  respondents. 


9.2.3    Computing  Plausible  Values  in  IRT  based  Scales 

Plausible  values  for  each  respondent  i  are  drawn  from  the  conditional 
distribution  p(fi|Xi,yi).     This  subsection  describes  how,  in  IRT-based  scales, 
these  conditional  distributions  are  characterized,  and  how  the  draws  are 
taken.    Using  first  Bayes'  theorem,  then  the  IRT  assumption  of  conditional 
independence, 

p(e|xi,yi)  «  P(xjfl.yi)  p(fi|yi) 

-  P(xjn  p(fl|yi).  (9-5) 
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where  P(Xi|^)  is  the  likelihood  function  for  9  induced  by  observing 
(treating  item  parameter  estimates  as  knovm  true  values)  and  p(^|yi)  is  the 
distribution  of  $  given  the  observed  value       of  background  responses. 

Equations  (9.4)  and  (9.5)  can  also  be  employed  with  vector-valued  9,  as 
m  the  1936  NAEP  mathematics  subscales.     In  such  cases,  ?(x^\9)  is  the  product 
over  subscales  of  the  independent  likelihoods  induced  by  responses  to  J^'^ems 
within  each  subscale,  and  p(^|yi)  is  the  multivariate — and  generally  non- 
independent — Joint  density  of  proficiencies  for  the  subscales,  conditional  on 
background  variables  y. 

In  the  analyses  of  1988  NAEP  data,  a  normal  (Gaussian)  form  was  assumed 
for  p(^|yi),  with  a  common  dispersion  and  with  a  mean  given  by  a  linear  model 
for  selected  main-effects  and  two-way  interactions  of  the  complete  vector  of 
background  variables.    The  included  background  variables  will  be  referred  to 
as  the  conditioning  variables,  and  will  be  denoted  y^.     (The  conditioning 
variables  used  in  1988  NAEP  analyses  are  listed  in  Appendix  C.)  Tlie 
following  model  was  fit  in  each  subject  area: 

^  -  r'  y^  +  e  ,  (9.6) 

where  c  is  normally  distributed  with  mean  0  and  dispersion  S.    T  and  S  are  the 
parameters  to  be  estimated.     In  subject  areas  witb  only  one  scale,  such  as 
reading,  F  is  a  vector  and  S  is  a  scalar.     In  subject  areas  comprising 
subscales,  T  i*»  a  matrix  and  S  is  a  covariance  matrix.    As  in  regression 
analysis,  F  is  a  vector  or  matrix  of  effects  and  S  is  the  matrix  or  scalar 
variance  of  residuals .    Also  as  in  regression,  the  interpretation  of  the 
effects  depends  on  how  the  design  vectors  in  y'^  are  coded — as  contrasts,  or 
for  linear  effects,  as  examples.    Appendix  C  gives  the  codings  and  estimates 
of  effects  of  the  present  assessment.     Like  item  parameter  estimates,  the 
estimates  of  the  parameters  of  conditional  oistributions  were  treated  ai,  known 
true  values  in  subsequent  steps  of  the  analyses. 

Maximum  likelihood  estimates  of  F  and  S  were  obtained  with  Sheehan's 
(1985)  M-GROUP  computer  program,  using  a  variant  of  the  riM  solution  described 
in  Mislevy  (1985).    The  difference  from  the  published  algorithm  lies  in  the 
numerical  approximation  that  was  employed.    Note  from  (9.5)  that  vi^l^i^Yi)  is 
proportional  to  the  product  of  two  terms,  the  likelihood  ?(x^\9)  and  the 
conditional  distribution  p(^|yi).     The  conditional  distribution  has  been 
assumed  multivariate  norwal,  with  mean  /ij  -  F'  yj  and  covariance  matrix  2,  if 
the  likelihood  is  approximated  by  another  normal  distribution,  with  mean  fi\ 
and  covariance  matrix  Sjf,  then  the  posterior  vi^l^i^Yi)  is  also  multivariate 
normal  with  covariance  matrix 

-  (S-^  +  (E^)-^)  (9.7) 
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and  mean  vector 


7i  -  (flf        +  t\  i:\  .  (9.8) 

In  the  1988  analyses,  a  normal  approximation  for  ?ix^\0)  is  accomplished 
in  a  given  scale  by  the  steps  described  below.     (Recall  that  by  the  assumed 
conditional  independence  across  scales,  the  joint  conditional  likelihood  for 
multiple  scales  is  the  product  of  independent  likelihoods  for  each  of  the 
irfcales.)     These  computations  are  carried  out  in  the  scale  determined  by  BTLOG 
(Mislevy  &  Bock,  1982)  item  parameter  estimates,  where  the  mean  and  standard 
deviation  of  the  composite  population  formed  by  combining  the  three  NAEP 
grade/ages  has  mean  zero  and  standard  deviation  one.    The  steps  were  as 
follows . 

1)  Lay  out  a  grid  of  Q  equally  spaced  points  from  -5  to  +5,  a  range 
that  covers  the  region  in  each  scale  where  all  examinees  from  all 
NAEP  grade/age  groups  are  virtually  certain  to  occur.    The  value 
of  Q  varies  from  20  to  40,  depending  on  the  scale  being  used; 
smaller  values  suffice  for  scales  with  few  items  given  to  each 
respondent,  while  larger  values  are  required  for  scales  with  many 
items. 

2)  At  each  point  X^,  compute  the  likelihood  L(Xi|5-Xq). 

3)  To  improve  the  normal  approximation  in  those  cases  in  which 
likelihoods  are  not  roughly  symmetric  in  the  range  of  interest — as 
when  all  of  a  respondent's  answers  are  correct — multiply  the 
values  from  Step  2  by  the  mild  smoothing  function 

exp(Xq+5) 

S(V  • 

[l+exp(Xq+5)]  [l+exp(:;q-5)] 

This  is  equivalent  to  augmenting  each  examinee's  response  vector 
with  responses  to  two  fictitious  items,  one  extraordinarily  easy 
item  that  ever3^one  gets  right  and  one  extraordinarily  difficult 
item  that    veryone  gets  wrong.    This  expedient  improves  the  normal 
approximation  for  examinees  with  flat  or  degenerate  likelihoods  in 
the  range  where  their  conditional  distributions  lie,  but  has 
negligible  effects  for  examinees  with  even  modestly  well- 
determined  symmetric  likelihoods. 

4)  Compute  the  mean  and  standard  deviation  of  6  using  the  weights 
S(Xq)L(Xi|5-Xq)  obtained  in  Step  3. 

At  this  stage,  then,  the  likelihood  induced  by  a  respondent's  answers  to 
the  items  in  a  given  scale  is  approximated  by  a  normal  distribution.  In 
subject  areas  where  there  is  only  one  scale,  a  single  normal  distribution  thus 
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sununarizes  information  from  item  responses.    In  an  area  such  as  mathematics  or 
science  where  there  are  several  scales,  independent  normal  distributions,  one 
per  subscale,  svimarize  information  from  responses  to  items  from  the  several 
scales. 

This  normalized- likelihood/normal  posterior  approximation  was  then 
employed  in  both  the  estimation  of  T  and  2  and  in  the  generation  of  plausible 
values.     From  the  final  estimates  of  F  and  2,  a  respondent's  posterior 
distribution  was  obtained  from  the  normal  approximation  using  the  four-s^ep 
procedure  outlined  above.    A  plausible  value  was  drawn  from  this  normal 
distribution — univariate  normal,  in  subject  areas  like  those  in  the  1988 
survey  with  only  a  single  scale;  multivariate  normal  in  areas  like  1986 
mathematics  and  science,  with  multiple  subscales.     For  those  subject  areas 
with  multiple  subscales,  weighted -average  composites  over  subscales  were  also 
calculated  after  appropriate  rescaling  (see  Beaton,  1983,  for  details  and 
definitions  of  composites). 

9.2.3.2         Computing  Plausible  Values  in  ARM  Scales 

In  1988  NAEP,  the  average  response  method  (ARM)  was  used  lo  carry  ^ut 
analyses  of  writing  data.    The  ARM  writing  composite  variable  is  defined  to  oe 
an  average  rating  (on  the  O-to-4  rating  scale  foi  responses  to  essay  pron;^ts) 
over  the  set  of  essay  exercises  detailed  in  Chapter  11.    Under  the  NAEP  hlE- 
spiraled  sampling  design,  no  single  student  is  administered  more  than  fo^r  of 
these  proap'cs.     It  is  possible  nevertheless  to  estimate  consistently  the 
covariance  among  each  pair  of  exercises,  setting  the  stage  for  the 
construction  of  plausible  values. 

The  key  step  in  the  scale -score  analyses  of  writing  w£.2>  the  creation  of 
sets  of  student-level  plausible  values.    If  a  respondent  had  answered  all 
questions  going  into  the  composite,  then  that  respondent's  ARM  score  would  be 
directly  calculable,  without  error,  by 

0  -  a'x 

where  x  is  the  vector  of  the  subject's  responses  to  the  n  questions  in  the 
composite  and  a  is  a  vector  of  n  constants,  each  equal  to  1/n.    However,  since 
no  respondent  was  presented  all  writing  exercises,  composite  values  must  be 
estimated  by  an  application  of  the  ARM  technology.     Briefly,  the  ARM 
technology,  which  is  a  kind  of  multiple  regression,  produces  for  each  studen*. 
a  set  of  plausible  values,  each  of  which  predicts  what  that  student's 
composite  score  might  plausibly  be,  based  on  the  student's  responses  to  the 
questions  in  the  composite  that  were  presented  to  the  student  and  based  on  the 
student's  status  on  the  conditioning  variables  listed  in  Appendix  C. 
("Conditioning  variables"  for  the  ARM,  just  as  for  IRT-based  plausible  values, 
are  the  background  variables  with  which  cognitive  responses  are  combined  to 
yield  respondents'  predictive  diiitributlons  for  9). 

Let  X|  represent  the  responses  of  the  i^^  student  to  the  questions  in 
the  composite  that  were  presented  to  that  student  and  let  y^  be  the  values  of 
that  student's  conditioning  variables.    Then  the  k^^  plausible  value  of  the 


240 


ARM  composite,  5,  based  on  the  student's  observed  responses  and  conditioning 
variables  isi 


A  A 

-  yi'  r+      )9+      7k+  ^i'  ^ik 

where 


is  the  k^*^  plausible  value  of  the  ARM  composite, 

is  the  vector  of  estimated  effects  for  the  conditioning 
variables, 

is  the  vector  estimated  as  giving  the  change  in  the  composite 
variable  for  a  unit  change  in  the  scores  on  each  of  the  questions 
in  xj,  ,*^ith  the  linear  effect  v^f  the  conditioning  variables  held 
fixed, 

is  a  random  draw  from  a  N(0,S)  distribution,  where  S  is  the 
estimated  variance-covariance  matrix  of  the  estimates  of  F  and 
P  and  reflects  the  uncertainty  due  to  using  sample  estimates  of 
the  regression  equation;  and 

is  an  estimated  residual  drawn  from  a  N(0,<7c)  distribution  where 
al  is  the  variance  of  the  predictive  distribution  of  the  ARM  value 
given  the  observed  values  of  y^  and  x^. 

The  parameters  relating  the  responses  on  a  given  set  of  background 
questions  (P)  and  values  of  the  conditioning  variables  (T)  with  the  means  of 
the  responses  each  of  the  questions  in  the  ARM  composite  were  estimated  by 
least-squares  technology.    To  accomplish  this  it  is  sufficient  to  obtain 
estimates  of  the  means,  variances,  and  interitem  covariances,  by  conditioning 
subgroup,  for  the  complete  set  of  writing  questions  going  into  the  composite. 
Because  the  ARM  composite  :     the  mean  of  the  individual  questions,  this  in 
turn  produces  estimates,  b^  conditioning  subgroup,  of  the  ARM  value  mean  and 
variance,  as  well  as  of  the  covariances  between  the  ARM  composite  and  each  of 
the  individual  vriting  questions.    These  provide  a  complete  set  of  sufficient 
statistics  (the  normal  equations)  for  the  standard  least-squares  prediction  of 
an  ARM  composite  value  given  conditioning  variable  characteristics  and 
responses  to  any  subset  of  the  writing  questions.     (See  Chapter  11  for  details 
of  the  gener/^tion  of  the  normal  equations.) 
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Solving  these  normal  equations  produces  the  standard  least-squares  point 
estimate  of  a  5:tudent's  score  on  the  composite,  which  is,  in  the  above 
notation, 

A  A  A 

h  -  yi'  r+  Xi'  . 

This  standard  estimate  does  not  take  into  accouwt  tue  distribution  of 
potential  scores  for  any  individual.    In  fact,  0^  is  an  estimate  ot  the  mean 
of  the  predictive  distribution  of  potential  0s  for  the  individual  and,  as 
such,  does  not  address  the  likelihood  ef  other  values  from  this  distribution, 
any  one  of  which  mignt  also  have  been  the  student's  ARM  composite  score  had 
the  student  answered  all  the  questions.     By  including  terms  the     account  for 
the  uncertainty  in  the  estimation  of  a  student's  composite  score,  the 
plausible  values  (Siy^)  provide  a  more  complete  representation  of  what  we  do 
and  do  not  know  about  the  student's  "true"  composite  score.     (Note:  For 
convenience  we  are  treating  the  ARM  composite  as  a  continuous  variable,  it  is 
in  fact  discrete,  but  can  take  c,  large  number  of  closely  spaced  values.) 

A  check  on  the  impact  of  the  approximations  and  simplifying  a^^jumptions 
employed  in  the  ARM  approach  was  carried  out  with  the  writing  data  from  the 
1984  NAEP  writing  assessment  (Beaton  &  Johnson,  1987).     As  a  comparison  for 
subgroup  average  writing  scores,  the  same  statistics  were  calculated  using  a 
totally  different  approach — the  model- free,  unbiased  estimate  for  average 
responses  based  on  the  methodology  employed  by  the  Education  Commission  of  the 
States  in  previous  NAEP  analyses.    The  latter  method  is  prohibitively 
expensive  to  be  used  for  all  NAEP  statistics,  but  could  be  calculated  for  the 
44  questions  in  the  common  background  questionnaire.     Beaton  and  Johnson  found 
that  statistics  based  on  the  ARM  were  nearly  indistinguishable  from  the  model - 
free  averages  for    hose  subgroups  distinguished  as  conditioning  variables,  and 
for  subgroups  whose  memberships  were  well-predicted  by  conditioning  variables. 
Estimatad  standard  errors  were  also  smaller  for  the  ARM  estimates.     For  those 
subgroups  that  were  neither  conditioned  on  nor  well-predicted  by  conditioning 
variables,  the  ARM  exhibited  biases.    The  nature  of  such  biases  in  plausible 
values  methodology  is  discussed  further  in  section  9.3.3  of  this  report. 
Their  causes,  properties,  and  remedies  are  discussed  at  length  in  Mislevy 
(1988a,  in  press; . 

9«3  ANALYSES 

When  survey  variables  are  observed  without  error  from  every  respondent, 
standard  variance  estimators  quantify  the  uncertainty  associated  with  sample 
statistics  from  the  only  source,  namely  the  sampling  of  respondents.  Item 
percents  correct  for  NAEP  cognitive  exercises  meet  this  requirement,  but 
scale-score  proficiency  values  do  not.    The  IRT  and  ARM  models  used  in  their 
construction  posit  an  unobservable  proficiency  variable  0  to  summarize 
performance  on  the  items  in  the  area.    The  fact  that  0  values  are  not  observed 
even  for  the  respondents  in  the  sample  requires  additional  statistical 
machinery  to  draw  inferences  about  0  distributions  and  to  quantify  the 
uncertainty  associated  with  those  inferences.    As  described  above,  we  have 
adapted  Rubin's  (1987)  multiple  iM>^tations  procedures  to  the  context  of 
latent  variable  models  to  produce^ife  ^plausible  values  upon  which  many 
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analyses  of  the  1988  NAEP  data  are  based.    This  section  describes  how 
plausible  values  were  e:s'  "*  *yecl  in  subsequent  analyses  to  yield  inferences 
about  population  and  sul       xlation  distributions  of  proficiencies. 


9.3.1    Computational  Procedures 

B  en  though  we  do  not  observe  the  6  value  of  respondent  i,  we  do  observe 
variables  that  are  related  to  it:    x^,  the  respondent's  answers  to  the 
cognitive  items  he  or  she  was  administered  in  the  area  of  interest,  and  y^, 
the  respondent's  answers  to  demographic  and  backg^-ound  variables.     Suppose  we 
wish  to  draw  inferences  about  a  number  T(i,X)  that  could  be  calculated 
explicitly  if  the  0  and  y  values  of  each  member  of  the  population  were  known. 
Suppose  further  that  if  6  values  were  observable,  we  would  be  able  to  estimate 
T  from  a  sample  of  N  pairs  of  6  and  y  values  by  the  statistic  t(i,2)  [where 
<£.,X)  ■  (^i.yi.  •  •  •  .^N»yK)l  »  ^^^^^  "®  could  estimate  the  variance  in  t  around 

T  due  to  sampling  respondents  by  the  function  U(i,x) .    Given  that  observations 
consist  of  (Xi,yi)  rather  than  (^i,yi)i  we  can  approximate  t  by  its  expected 
value  conditional  on  (x,2),  or 


t*  (x.x)  -  E[t(i,2)|x,2] 

-  ;  t(i,2)  p(i|x,x)  di  . 

It  is  possible  to  approximate  t*  with  randx^m  draws  from  the  conditional 
distributions  p(^i|xi,yi),  which  are  obtained^for  all  respondents  by  the 
method  describ  d  above  in  section  9.1.    Let       be  the  m*'*'  such  vector  of 
"plausible  values,"  consisting  of  a  (possibly  multidimensional)  value  for  the 
latent  variable  of  each  respondent.    This  vector  is  a  plausible  representation 
of  vjhat  the  true  S.  vector  might  have  been,  had  we  been  able  to  observe  it. 
The  following  steps  describe  how  an  estimate  of  a  scalar  statistic  t(i,2)  and 
its  sampling  variance  can  be  obtained  from  M  (>1)  such  sets  of  plausible 
values.     (Note:    Five  sets  of  ^^lausible  values  were  used  in  NAEP  analyses  in 
each  subject  area,  and  are  provided  on  the  NAEP  public -use  data  tapes  for 
secondary  analysis.) 

1)  Using  each  set  of  plausible  values  l^,  in  turn,  evaluate  t  as  if 
the  plausible  values  were  true  values  of  i.  Denote  the  results 
t„,  for  m-1, . . . ,M. 

2)  Using  the  multiple  weight  jackknife  approach  (see  Chapter  8), 
compute  the  estimated  sampling  variance  of  t„,  denoting  the  result 
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The  final  estimate  of  t  is 


M  ^ 

t*  -  S  t„  /  M  . 
m-1 

4)  Compute  the  average  sampling  variance  over  the  M  sets  of  pxausible 
values,  to  appr*^-ximate  uncertainty  due  to  suiopling  responoents- 

H 

-  S  U„  /  M  . 
m-1 

A 

5)  Compute  the  variance  among  the  M  estimates  t„,  to  approximate 
uncertainty  due  to  not  observing  B  values  from  respondents: 

M 

B„  -  2  (t„  .  t*)2  /  (M-1)  . 
m-'l 

6)  The  final  estimate  of  the  variance  of  t*  is  the  sum  of  two 
components : 

V  -  U*  +  (IfM'^)  Bj^  . 

Note:  Due  to  the  excessive  computation  that  would  be  required, 
NAEP  analyses  did  not  compute  and  average  jackknife  variances  over 
all  five  set?  cf  plausible  values,  but  only  on  the  first  jet. 
Thus,  in  NAEP  reports,  U*  is  approxiuated  by  U^. 


9.3.2    Statistical  Tests 

Suppose  that  if  6  values  were  obser"/ed  for  sampled  students,  the 
statistic  (t  -  T)/U^^^  would  follow  a  t-di&tribution  with  d  degrees  of 
freedom.    Then  the  incomplete-data  statistic  (t*  -  7)/V^^^  is  anproximatelv 
t-distributed,  with  degrees  of  freedom  given  by 


d  (l+r„)2 

 (M-1)  (l+r^;-^)2         d    , 

d  +  r„-2(M-l)  1  +  (d  r?.  /(m-D) 


where  r^^  is  the  relative  Increase  in  variance  due  to  not  obsi:.rving  B  values. 

r„  -  (1+m-^)  V  • 

When       is  small  relative  to  U*,  the  reference  distribution  for 
incomplete-data  statistics  differs  little  from  che  reference  distribution  for 
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the  corresponding  complete-data  statistics.  This  is  the  case  with  main  NAEP 
reporting  variables.  If  in  addition  d  is  large,  the  nr  nnal  approximation  can 
be  used  to  flag  "significant"  results. 

For  k-dimensional  t,  such  as  the  k  coefficients  in  a  multiple  regression 
analysis,  each  U^,  and  U*  is  a  covariance  matrix,  and  Bj^  is  an  average  of 
squares  and  cross-produzts  rather  than  simply  an  average  of  squares.     In  this 
case,  the  quantity 

(T-t*)  V'^  (T-t*)' 

is  approximately  F  distributed,  with  degrees  of  freedom  equal  to  k  and  i/,  with 
1/  defined  as  above  but  with  a  matrix  generalization  of  r^^: 

r^  -  d+M"^)  Trace  (B„U*"^)A. 

By  the  same  reasoning  as  used  for  the  normal  approximation  for  scalar  t,  a 
chi-square  distribution  on  k  degrees  of  freedom  often  suffices. 

9.3.3    Biases  in  Secondary  Analyses 

Statistics  t*  that  involve  proficiencies  in  a  scaled  content  area  and 
variables  included  in  the  conditioning  variables  y*^,  are  consistent  estimates 
of  the  corresponding  population  values  T.     Statistics  involving  background 
variables  y  that  were  not  conditioned  on,  or  relationships  among  proficiencies 
from  different  content  areas,  are  subject  to  asjmiptotic  biases  whose 
magnitudes  depend  on  the  type  of  statistic  and  the  strength  of  the 
relationships  of  the  nonconditioned  background  variables  to  tha  variables  that 
were  conditioned  on  and  to  the  proficiency  of  interest.    That  is,  the  large 
sample  expectations  of  certain  sample  statistics  need  not  equal  the  true 
population  parameters. 

The  direction  of  the  bias  is  typically  to  underestimate  the  effect  of 
nonconditioned  variables.     For  details  and  derivations,  the  interested  reader 
is  referred  to  Beaton  and  Johnson  (i987,  1990),  Mislevy  (1988a,  in  press),  and 
Mislevy  and  Sheehan  (1987,  section  10.3.5).     For  a  given  statistic  t* 
involving  one  content  area  and  one  or  more  nonconditioned  background 
variables,  the  magnitude  of  the  bias  is  related  to  (1)  the  extent  to  which 
observed  responses  x  account  for  the  latent  variable  6,  and  (2)  the      gree  to 
which  the  nonconditioned  background  var'.ables  are  explained  by  conditioning 
background  variables.    The  first  factor — conceptually  related  to  tast 
reliability — acts  consist  -.tly  in  that  greater  measurement  precision  reduces 
biases  in  all  secondary  analyses.    The  second  facuor  acts  to  reduce  biases  in 
certain  analyses  but  increase  it  in  others.     Jn  particular, 

•         High  shared  variance  between  conditioned  and  nonconditioned 

background  variables  mitigates  biases  in  analyses  that  involve 
only  proficiency  and  nonconditioned  variables,  such  as  marginal 
means  or  regressions. 
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©         High  shared  variance  exacerbates  biases  in  regression  coefficients 
of  conditional  effects  for  nonconditioned  variables,  when 
nonconditioned  and  conditioned  background  variables  are  analyzed 
jointly  as  in  multiple  regression. 

In  the  1984  NAEP  reading  assessment,  the  magnitude  of  shrinkage  for  the 
subgroup      ans  of  a  background  variable  that  was  not  conditioned  on  averaged 
about  15  percent.    Biases  in  multiple  regressions  that  included  conditioning 
variables  averaged  about  35  percent.    Since  that  time,  two  importar  steps 
have  been  taken  to  greatly  reduce  potential  biases  of  this  tjrpe.    Fiist  is  the 
move  to  the  "focused -BIB"  matrix- sa,.pling  design,  under  which  all  the 
cognitive  tasks  a  respondent  is  administered  are  drawn  from  the  same  subject 
area.    On  the  average,  respondents  are  presented  about  twice  as      ^^y  tasks  in 
the  subject  area  than  would  have  been  presented  under  the  full  spiraling 
design,  which  administered  each  examinee  tasks  from  one,  two,  or  three  subject 
areas.    This  increases  the  extent  to  which  x  accounts  for  9,  and,  as  noted 
above,  decreases  potential  biases  in  all  secondary  analyses.     Second  is  the 
increase  in  the  number  of  background  variables  that  can  be  included  in  the 
conditioning  vector.    Tliis  increases  the  number  of  secondary  analyses  that  can 
be  carried  out  with  little  or  no  bias,  and  mitigates  biases  in  analyses  of  the 
marginal  distributions  of  9  in  nonconditioned  variables.    Bruce  Kaplan  and 
Jennifer  Nelson's  analyses  of  1988  reading  data  (some  results  of  which  are 
summarized  in  Mislevy,  in  press)  indicate  that  these  improvements  have  slashed 
the  potential  bias  for  nonconditioned  variables  in  multiple  regression 
analyses  from  the  1984  level  of  35  percent  to  approximately  10  percent,  and 
biases  in  simple  regression  of  such  variables  from  15  percent  to  5  percent. 

Table  9-1  gives  representative  results  from  an  analysis  in  which  Nelson 
estimated  a  number  of  substantively  important  effects  from  the  1988  NAEP 
reading  data  for  13-year-olds  in  two  ways:  (1)  with  the  operational 
conditioning  process,  which  included  the  listed  effects  in  the  conditioning 
vector,  so  their  estimates  are  consistent;  and  (2)  with  no  conditioning  at 
all,  so  that  the  biases  would  be  at  their  maxima.    Also  shown  are  results  of 
an  experimental  method  of  conditioning,  which  employed  only  the  first  32 
principal  components  of  the  matrix  of  all  64  original  conditioning  vectors. 
The  encouraging  result  from  this  analysis  is  that  bias  in  analyses  involving 
the  original  effects  is  virtually  eliminated  in  this  approach  through  the  use 
of  only  half  as  many  conditioning  variables. 
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Table  9-1 

Estimated  Effects  Based  on  Full,  No,  and  Partial  Conditioning 


Conditioning 


Effect 

Full* 

None 

Bias 

32  Componants 

Bias 

Male-Female 

1  R  "7 

1  A  A 

-o* 

-IS  Q 

-  J.  J .  y 

White -Black 

26.6 

23.8 

-IIZ 

26.3 

-u 

High  Metropolitan- Low 
Metropolitan 

32.6 

30.5 

-6X 

32.8 

IX 

Northeast- Southeast 

10.4 

9.4 

-lOX 

10.1 

-3X 

13 -year-old  Eighth  Graders - 
13-ye;ar-old  Seventh  Graders 

32.7 

29.2 

-11% 

32.7 

0 

shown  hcire 


*  Imputations  constructed  with  conditional  distributions  that  included  6A  contrasts,  including  those 


9.3.4 


A  Ntimerical  Example 


To  illustrate  how  plausible  values  are  used  in  subsequent  analyses,  this 
subsection  gives  some  of  the  steps  in  the  calculation  of  1988  grade -level 
reading  means  and  their  estimation-error  variances. 

The  weighted  mean  of  the  first  plausible  values  of  the  grade  4  students 
in  the  sample  is  230.68,  and  the  jackknife  variance  of  these  values  is  1.17. 
Were  these  values  true  0  values,  then  230.68  would  be  the  estimate  of  the  mean 
and  1.17  would  be  the  estimation-error  variance     The  weighted  mean  of  the 
secoid  plausible  values  of  the  same  students,  however,  is  23C  60;  the  third, 
fourth,  and  fiith  plausible  values  give  weighted  means  of  230.19,  230.32,  and 
230.06.    Since  all  of  these  figures  are  based  on  precisely  the  same  sample  of 
studencs,  the  variation  among  them  is  due  to  uncertainty  about  the  students' 
0s,  having  observed  their  item  responses  and  background  variables.    Taking  the 
jackknife  variance  estimate  from  the  first  plausible  value,  1.17,  as  our 
estimate  U*  of  sampling  variance,  and  the  variance  among  the  five  weighted 
means,   .09,  as  our  estimate  B  of  uncertainty  due  to  not  observing  6,  we  obtain 
as  the  final  estimate  V  of  total  error  variance  1.17  +  (1+5"^)  .09  -  1.28. 

With  U*  and  B  defintd  as  above,  and  wi:h  M-5,  we  may  obtain  a  value  for 
Rubin's  (1987)  index  characterizing  the  relative  increase  in  variance  due  to 
the  latency  o.*"       r  -  .09. 


Corresponding  values  were  also  calculated  for  grade  8  and  grade  12, 
results  are  shown  in  Table  9-2. 


The 
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Table  9-2 

Estimation  Error  Variances  and  Related  Coefficients 
for  the  1988  Grade- level  Reading  Assessments 


Grade 


B 


V 


r 


4 
8 
12 


1.17 
.96 
.69 


.09 
.06 
.02 


1.28 
1.03 
.71 


.09 
.07 
.03 


9.4      OVERVIEW  OF  SCALES  IN  THE  1988  NAEP  ASSESSMENT 

Scale- score  analyses  were  carried  out  in  t'ae  following  subject  areas  in 
^  i  1988  NAEP  assessment. 


Reading:     1  IRT  trend  scale,  linking  1988  results  to  the  1971  - 
1984  assessments,  and  1  IRT  cross-sectional  scale,  newly  created 
in  1988 

Writing:     1  ARM  trend  scale,  linking  1984  and  1988,  and  1  ARM 
cross- sectional  scale,  newly  created  ir>.  1988 

Civics:     1  IRT  trend  scale,  linking  1988  results  to  the  1976  and 
1982  assessments  (neither  of  vhich  had  been  previously  scaled), 
and  1  IRT  cross-sectional  scale,  newly  created  in  1988. 

U.S.  history:    1  IRT  scale,  newly  created  in  1988. 

Geography:     1  IRT  scale,  newly  created  in  1988. 


One  subset  of  conditioning  variables  was  employed  in  the  creation  of 
plausible  values  for  all  of  these  areas;  it  included  only  variables  from  the 
common  set  of  background  questions  administered  to  respondents  in  all  subject 
areas.    The  variables  involved  are  listed  in  Table  C-1  in  Appendix  C.  A 
second  subset  of  subject-specific  ccnditioning  variables  was  additionally 
included  in  the  creation  of  plausible  values  for  the  subject  areas  of  reading, 
writing,  civics,  U.S.  history,  and  geography;  these  subsets  included  variables 
based  on  subject- related  questions  administered  only  to  respondents  who  were 
administered  cognitive  items  in  the  corresponding  area.    Tables  C-2  through  C- 
11  in  Appendix  C  give  the  relevant  variables  and  details  of  exactly  how  the^ 
background  effects  were  coded  in  order  to  produce  the  conditioning  vector  y" 
Conditional  effect  parameters  T  and  the  associated  residual  variances  S  were 
estimated  separately  in  each  subject  area  and  in  each  grade/age  Estimated 
effects  for  each  subject  and  sample  are  ci^en  ^n  Tables  C-12  through  C-3/. 
Additional  information  on  these  analyses  is  presented  in  Chapters  10  through 
14. 

In  the  course  of  administering  bridge  (or  trend)  sample  test  forms 
comparable  to  those  of  the  1986  survey  to  further  study  trends  in  reading, 
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data  were  s<»rendipitously  obtained  for  mathematics  and  science.  Supplementary 
analyses  involving  these  data  are  described  in  Chapter  15.    These  analyses 
included  scale-score  analyses  as  follows: 

•  Mathematics:    1  IRT  scale,  linked  with  the  mathematics  trend  scale 
created  in  1986. 

•  Science:    1  IRT  scale,  linked  with  the  mathematics  trend  scale 
created  in  1986. 

The  conditioning  variables  and  codings  used  in  the  creation  of  plausible 
values  for  these  two  subject  areas  are  listed  in  Tables  C-10  and  C-11  in 
Appendix  C;  estimated  effects  for  the  mathematics  and  science  conditioning 
variables  are  given  in  Tables  C-38  through  C-43. 
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Chapter  10 

DATA  ANALYSIS  FOR  THE  READING  ASSESSMENT^ 


Rebecca  Zwick 
Educp*-ional  Testing  Service 


The  analysis  of  the  1988  reading  data  had  three  main  goals: 

•         to  continue  the  long- term  reading  trend  results  begun  in 
1971  for  ages  9,  13,  and  17, 

to  produce  detailed  analyses  of  reading  proficiency  for 
grades  4,  8,  and  12,  and 

t         to  contin'^e  the  investigation  of  the  anomalous  reading 
results  in  1986.^ 

The  details  of  the  first  two  t  ,jes  of  analyses  are  given  in  this  chapter.  The 
1988-1989  investigation  of  tl  1986  reading  anomaly  is  described  in  The  Effect 
of  Changes  Ln  the  National  Assessment:  Disentangling  the  NAEP  1985-86  Reading 
Anomaly  (Beaton  &  Zwick,  1990). 

This  chapter  has  three  main  sections.     First,  the  samples  of  stud^.nts 

who  received  rea.  ing  items  in  the  1988  NAEP  assessment  are  described.  Th?n, 

the  procedures  used  for  the  long-term  rrend  analysis  are  detailed.  Finally, 

the  procedures  used  for  the  cross-sectional  analysis  are  discussed. 


^  David  Freund  provided  statistical  programming,  with  the  assistance  of 
Mirihwei  Wang  and  Kate  Pashley.    Robert  Mislevy  provided  consultation  on 
scaling.     Jo-Ling  Liang  assisted  with  analyses. 

^In  1986,  reading  trend  results  for  ages  9  and  17  appeared  implausibly 
low.    Their  release  was  therefore  delayed  and  a  three-year  investigation 
ensued.     (Despite  the  la.        hange  in  reading  proficiency  estimates,  the  1986 
findings  res^.mble.d  past  assessments  and  other  reading  measures  with  respect  to 
the  ordering  of  subgroup  differences  and  th^  relation  of  reading  proficiency 
to  background  variables  and  to  performance  in  mathematics  and  science.     It  was 
therefore  decided  to  release  a  cross-sectional  report.  Who  Reads  Best? 
[Applebee,  Langer,  &  Mullis,  1988],  using  a  scale  metric  that  differed  from 
the  1984  reading  scale.)    The  anomalous  1986  reading  trend  results  are  not 
included  in  NAEP's  reading  trend  report,  but  are  documented  separately  in 
Beaton  and  Zwick,  1990,  whi  h  describes  the  investigation  of  the,e  results  and 
their  subsequent  adjustment.     In  brief,  the  study  showed  that  changes  in 
assessment  technology  had  affected  the  1986  estimates  of  reading  proficiency 
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10.1    SAMPLES  OF  STUDEN^IS 


In  1988,  reading  items  were  administered  to  14  samples  of  students  in 
the  1988  assessment,  as  shown  in  Tabl^  10-1: 

#         The  spjnples  that  constituted  the  bridge  to  1984  (9[Br84-RW], 

13[Br84-RW],  and  17[Br84-RW])  played  a  dual  role  in  the  1988  reading 
analyses:    They  served  as  the  basis  of  the  long-term  reading  trend 
estimates,  reported  in  The  Reading  Report  Card,  1971  to  1988:  Trends 
from  the  Nation's  Report  Card  (Mullis  &  Jenkins,  1990)  and  were  used  in 
the  1988-1989  phase  of  the  investigation  of  the  1986  reading  anomaly. 


•  The  "focused-BIB"  samples  for  reading  (i.e.,  the  balanced  incomplete 
block  spiral  samples  9[Main-Rdg],  13[Main-Rdg] ,  and  17 [Main-Rdg] ,  which 
received  three  reading  blocks)  served  as  the  basis  of  the  grade-level 
cross-sectional  analyses  reported  in  Learning  to  Read  in  Our  Nation's 
Schools:     Instruction  and  Achievement  in  1988  at  Grades  4,  8,  anc*  12 
(Langer,  Applebee,  Mullis,  &  Foertsch,  1990). 

•  The  intercorrelation  samples,  9[Main-Int],  13 [Main-Int] ,  and 
17[Main-Int] ,  which  were  also  a  part  of  the  main  assessment,  received 
booklets  that  contained  reading,  civics,  and  U.S.  history  items,  and,  at 
grade  12/age  17,  geography  items.    The  scaling  of  the  results  from  these 
samples  allows  researchers  to  examine  the  relation  between  these  subject 
areas. 


•         For  grade  8/age  13  and  grade  12/age  17,  the  main  assessment  also 

included  samples  of  students  (9 [Main-Doc]  and  13 [Main-Doc])  who  received 
two  booklets  consisting  of  NAEP  reading  blocks  along  with  docuipent 
literacy  items  that  had  been  administered  as  part  of  NAEP's  1985  study 
of  adult  literacy  (Kirsch  &  Jungeblut,  1986).    These  booklets  differed 
f^om  the  remaining  main  NAEP  booklets  for  grade  8/age  13  and  grade 
12/age  17  in  that  the  grid  of  response  ovals  for  the  items  followed 
immediately  after  each  item  in  the  booklet;  for  the  remainder  of  the 
main  NAEP  samples  for  the  two  upper  age  classes,  a  separate  scannable 
answer  sheet  was  used.     (Tabla  10-2  shows  the  response  modes  for  all 
portions  of  the  1988  reading  assessment.)    Preliminary  item  analyses 
showed  that  when  reading  items  were  administered  in  these  document 
literacy  booklets,  they  behaved  in  a  substantially  different  manner  than 
in  the  focused-BIB  assessment.    Therefore,  the  reading  item  responses 
from  these  booklets  were  not  scaled,  but  are  available  for  secondary 
analysis.    Item  results  for  the  document  literacy  blocks  (which, 
unfortunately,  showed  that  large  proportions  of  students  failed  to  reach 
some  items)  are  also  available  on  NAEP  user  tapes. 
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Table  10-1 
NAEP  1988  Reading  Samples 


Subject 

Booklet 

Time  of 

Age 

Modal  Sampl 

Sample  Code 

Areas 

Numbers 

Cohort 

Assessment 

Defn. 

Grade 

Size 

9{Br84-RW] 

R, 

w 

51-56 

Grade  4/age  9 

Winter 

CY 

4 

5,188 

13[Br84-RW] 

R. 

w 

51-56 

Grade  8/age  13 

Fall 

CY 

8 

5,500 

17[Br84-RW] 

R. 

w 

51-56 

Grade  11/age  17 

Spring 

not  cy 

11 

4  622 

9[Main-Rdg] 

R 

8-14 

Grade  4/age  9 

Winter, 

spring 

CY 

4 

6,177 

13[Main-Rdg] 

R 

8-14 

Grade  8/age  13 

Winter, 

spring 

CY 

8 

5,912 

17[Main-Rdg] 

R 

8-14 

Grade  12/age  17 

Winter 

SDr in? 

CY 

12 

5,768 

9[Main-Int] 

R. 

C,  H 

17-19 

Grade  4/age  9 

Winter, 

spring 

CY 

4 

2,638 

13[Main-Int] 

R, 

C,  H 

29-31 

GraHA  8/age  13 

Winter, 

spring 

a 

A 

2  590 

17[Main-Int] 

R, 

C,  H,  G 

30-32 

Graae  12/age  17 

Winter, 

spring, 

CY 

12 

A* ,  ^ 

13  [Main-Doc] 

R. 

D 

35-36 

G  ade  8/age  13 

Winter, 

spring 

CY 

8 

2,533 

17  [Main-Doc] 

R. 

D 

36-37 

Grade  12/age  17 

Winter, 

spring 

CY 

12 

2,425 

9[Br86-RMS] 

R. 

M,  S 

9jl-93 

Age  9 

Winter 

CY 

4 

3,711 

13[Br86-RMS] 

R. 

M,  S 

91-93 

Age  13 

Fall 

CY 

8 

3,942 

17[Br86-RMS] 

R, 

M,  S 

61-66 

Grade  11/age  17 

Spring 

not  CY 

11 

4,703 

R  -  Reading 

W  -  Writing 

C  -  Civics 

H  -  U.S.  history 

G  -  Geography 

D  -  Document  Literacy 


M  -  Mathematics 
S  «  Science 

CY  -         Calendar  year:  bi>-th  dates  in  1978,  197  , 
and  1970  for  ages  9,  13,  and  17 

not  CY  «  (Age  17  only) :  birth  dates  between  October 
1,  1970  and  September  30,  1971 
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Table  10-2 


Response  Modes  for  NAEP  1988  Reading  Items 


Sample  Code 

9[Br84-RW] 
13(Br84-RW] 
17[Br84-RW] 

9[Main-Rdg] 
13(Main-Rdg] 
17[Hain-Rdg] 

9[Main-Int] 
13[Maxn-Int] 
17[Mpin-Int] 

13[Maln-Doc] 
17[Main-Doc] 

9(Br86-RMS] 
13[Br86-RMS] 
17[Br86-RMFl 


Sample  Type 

Bridge  to  1984 
Bridge  to  1984 
Bridge  to  1984 

BIB  Main 
BIB  Main 
BIB  Main 

BIB  Intercorrelation 
BIB  Intercorrelation 
BIB  Intercorrelation 

BIB  Docximent  Literacy 
BIB  Docximent  Literacy 

Bridge  to  1986 
Biidge  to  1986 
Bridf     to  1986 


Booklet 
Numbers 

51-56 
51-56 
51-56 


Response  Mode 

Circle  response  in  booklet 
Circle  response  in  booklet 
Circle  response  in  booklet 


8-14 

Fill 

in 

oval 

in 

booklet 

8-14 

Fill 

in 

oval 

m 

separate 

8-14 

Fill 

in 

ovai 

>n 

separate 

17-19 

Fill 

in 

oval 

in 

booklet 

29-31 

Fill 

in 

oval 

on 

separate 

30-32 

Fill 

in 

oval 

on 

separate 

35-36 

Fill 

in 

oval 

in 

booklet 

36-37 

Fill 

in 

oval 

in 

booklet 

91-93 

Fill 

in 

ove 

in 

booklet 

91-93 

Fill 

in 

oval 

in 

booklet 

61-66 

Fill 

in 

oval 

in 

booklet 
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•         The  reading  blocks  in  the  bridge-to-1986  samples,  9[Br86-RMS], 

13[Br86-RMS] ,  and  17 [Br86-RMS] ,  were  used  only  in  the  analysis  of  the 
reading  anomaly  and  are,  therefore,         discussed  further  in  this 
report. 


10.2     LONG-TERM  TREND  ANALYSIS— BRIDGE  TO  1984  SAMPLES 

The  1988  bridge  to  1984  included,  at  each  age  level,  six  of  the 
assessment  booklets  administered  in  1984.    These  booklets  contained  both 
reading  and  writing  blocks,  as  well  as  background  items.    Although  these 
bridge  booklets  represented  only  about  a  tenth  of  the  reading  booklets 
administered  in  the  complex  1984  BIB  design,^  they  contained  10  of  the  12 
reading  blocks  that  were  scaled  at  each  grade/age  level  in  1984.    The  number 
of  students  who  were  given  each  of  the  bridge  items  ranged  from  768  to  927. 
The  samples  of  students  who  received  these  bridge  booklets  are  described  in 
Table  10-1  and  in  Chapter  3.    The  purpose  of  the  long-term  reading  trend 
analysis  was  to  add  to  the  reading  trend  results  that  extended  from  1971  to 
1984  for  ages  9,  13,  and  17.*    Dimensionality  analyses  conducted  following  the 
1984  assessment  showed  that  the  reading  items  were  well  summarized  by  a 
unidimensional  scale  (Zwick,  1987a).    The  analysis  steps  were  as  follows: 


1.     Conduct  preliminary  item  analyses  and  verify  the  fit  of  the  1984 
item  parameters  to  the  1988  bridge  data.     For  each  item,  calculations  were 
made  of  the  percent  of  students  selecting  each  response,  the  percent  who 
omitted  the  item,  the  percent  who  did  not  reach  the  item,  and  the  correlation 
between  the  item  score  and  the  block  score.    Also,  for  each  item  block,  the 
internal  consistency  reliability  was  computed.    The  block-level  KR-20 
reliabilities  (for  scaled  multiple-choice  items  only)  ranged  from  .66  to  .83, 
with  a  median  of  .74,  at  age  9;  from  .46  to  .74,  with  a  median  of  .64,  at  age 
13;  and  from  .31  (for  a  block  with  only  four  scaled  items)  to  .75,  with  a 
median  of  .67,  at  age  17.    The  item  analyses  revealed  that  at  least  10  percent 


^The  bridge  to  1984  included  1984  booklets  16,  17,  27,  34,  55,  and  60  at 
age  9  and  booklets  13,  16,  17,  21,  34,  and  57  at  ages  13  and  17  (see  J.  R. 
Johnson,  1987,  pp.  120-121).    The  1984  BIB  assessment  included  57  booklets 
that  contained  at  least  one  scaled  reading  block  at  age  9  and  56  such  booklets 
at  ages  13  and  17 . 

*Note  that  the  current  estimates  of  reading  proficiency  results  for  1971 
through  1984  differ  slightly  from  those  that  appear  in  The  Reading  Report  Card 
(NAEP,  1985)  for  three  reasons.    The  first  is  that  an  improvement  in  the 
capacity  of  the  computer  program  used  for    the .conditioning  phase  of 
proficiency  estimation  (see  Chapter  9)  allowed  for  an  increase  in  the  number 
of  conditioning  variables,  leading  to  improved  proficiency  estimates  for  the 
years  1971  through  1984.    Also,  additional  records  for  1971  that  had  been 
omitted  from  tapes  provided  by  the  previous  contractor  were  recovered.  The 
third  change  resulted  from  a  recomputation  of  the  1984  weights  for  grade  4/age 
9  and  grade  8/age  13.    This  change  is  detailed  in  Appendix  E. 
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of  the  students  failed  to  reach  22  items  at  age  9,  seven  items  at  age  13,  and 
one  item  at  age  17.    Unlike  the  main  NAEP  assessment,  however,  the  bridge 
blocks  cannot  be  altered  to  reduce  speededness.     Evidence  of  speededness  in 
the  main  assessment  is  presented  below. 

Because  the  purpose  of  the  analysis  of  the  bridge  data  was  to  provide  a 
link  to  the  1984  reading  results,  it  was  desirable  to  use  the  parameters  that 
had  been  obtained  for  the  bridge  items        the  1984   .tem  calibration,  provided 
that  the  fit  of  the  1984  parameters  to  the  1988  data  was  acceptable.  (Hie 
1984  analysis  included  all  three  grade/age  cohorts  in  a  single  calibration.) 
In  1988,  all  multiple-choice  items  were  included  in  the  scaling  process.  There 
were  99  such  items  at  ages  9  and  13  and  87  at  age  17.     The  total  number  of 
scaled  items  was  182.  (The  decision  to  exclude  open-ended  items  was  influenced 
by  consideration  of  the  role  of  the  bridge  data  in  the  reading  anomaly 
investigation:  Because  a  previous  study  (Zwick,  1988]  had  shown  that  scoring 
inconsistencies  had  affected  these  items,  their  inclusion  could  have 
complicated  the  study  of  the  reading  aiiomaly.)    Out  of  the  182  items,  65  items 
were  administered  to  grade  8/age  13  and  grade  12/age  17,  37  items  were  given 
to  grade  4/age  9  and  grade  8/age  13,  25  items  were  given  to  grade  4/age  9  and 
grade  12/age  17,  and  24  items  were  given  to  all  three  age  classes. 

For  all  182  items  to  be  included  in  the  1988  scaling,  plots  of  the  1984 
item  response  functions  along  with  the  1988  data  showed  the  fit  of  the  1984 
items  to  the  1988  data  to  be  acceptable.    These  item  parameters  are  given  in 
Table  F-2  of  Appendix  F. 

2.    Obtain  the  proficiency  means  and  standard  errors,  and  percencs  of 
students  above  each  scale  anchoring  point  for  the  NAEP  reporting  groups. 
Although  scale  values  were  obtained  for  the  grade/age  samples,  only  the  age 
samples  were  used  for  trend  rerjorting,  following  the  NAEP  tradition.  Sample 
sizes  were  3,782,  4,005,  and  3,652  for  ages  9,  13,  and  17.     Foi:  each  of  the 
NAEP  reporting  categories,  reading  proficiency  mi.ans  and  standard  errors  were 
computed  using  the  technology  described  in  Chapter  9.    A  list  of  the 
conditioning  variables,  the  scheme  for  coding  them,  and  their  estimated 
effects  appear  in  Tables  C-19  to  C-21  of  Appendix  C.     In  addition,  for  each 
reporting  category,  a  determination  was  made  of  the  percent  of  students 
exceeding  each  of  ^he  scale  anchoring  levels  determined  in  1984:  Rudimentary 
(150),  Basic  (200),  Intermediate  (250),  Adept  (300),  and  Advanced  (350).  The 
methods  for  deriving  the  anchor  points  is  outlined  in  Chapter  7  and  described 
in  detail  in  Beaton  (1987a). 


10.3    CROSS -SECTIONAL  ANALYSIS— MAIN  SAMPLES 

In  1988,  reading  items  were  administered  in  seven  focused  BIB  booklets, 
each  of  which  contained  three  reading  blocks,  in  addition  to  background  items. 
The  samples  of  students  who  received  these  items  are  listed  in  T ible  10-1  and 
described  m  Chapter  3.     In  addition,  at  each  grade/age  level,  one  reading 
block  was  included  in  each  of  the  three  booklets  administered  to  the 
intercorrelation  samples.    The  numbers  of  reading  items  administered  to  *e 
main  samples  are  given  in  Table  10-3,  along  with  th3  numbers  of  items  that 
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were  new  in  1988,    Reading  objectives  for  the  new  items  are  documented  in 
Reading  Objectives,  1986  and  1988  Assessments  (NAEP,  1987a;  see  Chapter  2). 


Sample  Code 

9[Main-Rdg] 
13[Main-Rdg] 
17{Main-Rdg] 

Total 


Table  10-3 
NAEP  1988  Reading  Items — ^BIB  Samples 

Number  of  Number  of 

P-^>adinp  Scale  Items         Reading  Scale  Items  New  to  1988 


83 
100 
110 

215 


43 
53 
66 

126 


Of  the  total  of  215  items,  52  items  (including  three  intact  blocks)  were 
administered  to  both  grade  8/age  13  aud  grade  12/age  17,  26  items  were  given 
to  both  grade  V^ge  9  and  grade  8/age  13,  and  5  items  were  given  to  all  three 
age  classes.    The  number  of  students  in  the  focused-BIB  and  intercorreiation 
samples  who  were  given  each  item  ranged  from  2,459  to  3,536. 

Four  of  the  215  items  were  open-ended;  the  remainder  were  multiple- 
choice.    Two  of  the  four  open-ended  items  were  administered  at  grade  li/age  1 
only,  one  at  grade  4/age  9  only  and  one  at  both  grade  4/age  9  and  grade  8/ag 
13.    These  items  were  rated  on  ordinal  scales  by  professional  judges.  Scoring 
procedures  and  reliability  results  are  described  in  Chapter  6.2.     For  purposes 
of  item  response  theoiy  (IRT)  scaling,  the  range  of  possible  scores  was  then 
dichotomized  into  "correct*'  and  "incorrect"  categories  using  rules  provided  by 
reading  expert^;.     Table  10-4  shows  the  number  of  points  in  the  ordinal  scale 
for  each  item,  along  with  the  dichotomization  rule. 


Table  10-4 

Dichotomization  Rules  for  Open-ended  Reading  Items  Used  in  Scaling 


NAEP  Item  Grade /Age 

N015905  High  Tech  Pizza  12/17 

R002406  Small  Fruits  12/17 

R000106  Ant  and  Dove  'V9 

R000806  Grandpa  and  Wind  4/9,  8/13 


Score  Range  for 
Valid  Responses 

1  - 

1  -  2 

1  -  5 

1  -  5 


Scores 
Considered 
Correct 

2  -  4 
2 

4  -  5 

3  -  5 


The  primary  purpose  of  the  analysis  of  tlie  main  NAEP  data  was  to  provide 
reading  results  for  grades  4,  8,  and  12,     nd  to  investigate  the  relation  of 
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reading  proficiency  to  student  background  and  attitudes  and  to  teacher 
attributes  and  instructional  methods.      The  analysis  steps  were  as  follows. 

1.    Conduce  item  analyses.    Item  analysis  procedures  like  those 
described  in  connection  with  the  trend  analysis  were  applied  to  the  main  NAEP 
blocks,     block-leval  KR-20  reliabilities  (for  multiple-choice  items  only; 
ranged  from  .62  to  .84,  with  a  median  of  .81,  at  grade  4;  from  .66  to  .87, 
with  a  median  of  .84,  at  grade  8;  and  from  .74  to  .85,  with  a  median  of  .81, 
at  grade  12.    An  important  findii.g  of  the  item  an£.  yses  was  that,  although 
NAEP  assessments  are  not  intended  to  be  speeded,  some  of  the  main  NAEP  reading 
blocks,  like  the  bridge  blocks,  tended  to  be  too  long  for  substantial  numbers 
of  examinees.    At  least  10  percent  of  the  students  failed  to  reach  13  items  at 
grade  4,  10  icems  at  grade  8,  and  three  items  at  age  17.     The  percent  of 
students  who  failed  to  reach  (or  chc3e  not  to  respond  to)  the  open-ended 
items,  which  are  placed  at  the  end  of  reading  blocks  in  NAEP,  was  always  at 
least  20.    Tlie  information  gained  from  the  1988  assessment  about  the  numbers 
of  items  that  students  can  complete  was  used  in  determining  the  number  of 
items  for  the  1990  item  blocks.     It  is  hoped  that,  by  reducing  the  length  of 
blocks,  spaededness  problems  can  be  avoided  in  the  future. 

It  was  also  discovered  in  examining  the  reading  rejults  that  the 
performance  on  each  item  block  was  affected  by  the  position  of  the  block 
within  the  booklet.    Table  10-5  shows  the  average  percent  of  student?  who 
failed  to  reach  the  items  in  each  block  for  each  of  the  three  possible  block 
positions.     In  nearly  <     ry  case,  the  average  percent  not  reached  is  greater 
for  position  3  than  for    ositions  1  and  2.     Particularly  notable  are  the  large 
average  percents  not  reached  for  the  grade  4  blocks  R3  and  R7  in  positi.  ^  3. 
The  average  percent  correct  for  each  block  in  each  of  the  three  possible 
positions  is  shown  in  Table  10- o.    In  nearly  every  case,  the  average  percent 
correct  declines  as  the  position  moves  from  1  to  3.     The  effect  of  block 
position  is  most  severe  for  grade  4,  followed  in  order  by  grade  8  and  grade 
12.     The  effect  is  most  striking  for  block  R3  at  grade  4,  for  which  the 
ftVera^e  percent  correct  for  position  1  is  n'^arly  11  percent  points  higher  than 
for  position  3.    In  NAEP's  computation  of  the  percent  correct  for  an  item, 
students  who  d^''.  not  reach  the  item  are  not  included';  therefore,  these 
differences  in  average  percents  correct  are  not  simply  the  result  of  students' 
failure  to  reach  items  that  occur  in  later  blocks.     Perhaps  fatigue  leads  to 
more  errors  in  these  blocks.     Further  investigation  jf  this  issue  is  planned. 


ii  ^Percent  correct  is  defined  as  R/(R  +  W  +  0  +  DK) ,  where  R,  W,  0,  and  DK 

represent  the  sura  of  the  student  weights  for  those  who  got  the  ,  .  en>  right, 
those  who  got  the  item  wrong,  those  who  reached  the  item  but  caiittfcd  and 
those  who  indicated  that  they  did  not  know  the  answer,  respectively.     The  DK 
option  is  included  only  in  certain  trend  items.     Students  who  did  not  reach 
the  item  are  not  included  in  the  computation.    An  item  is  considered  "uot 
reached"  if  the  student  did  not  respond  to  the  item  and  did  not  give  a  valid 
response  to  any  of  the  succeeding  items  within  tie  item  block.     Each  block  of 
items  is  separately  timed  and  therefore,  the  determination  of  which   .tems  are 
to  be  considered  "not  reached"  is  made  separately  by  block. 
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Toile  10-5 


Average  Percent  of  Students  Failing  to  Rear.h  Items 
for  Each  Block  Position 


Block* 


Grade  4 


Number  of 
Items 


Position 
1 


Position 

2 


Position 
3 


R2 
R3 
R4 
R5 
R6 
R7 
R8 


14 
7 
7 
15 
14 
15 
12 


5.8 
4.7 
2.4 
5.5 
5.0 
3.7 
4.6 


3.0 
4.8 
2.2 
4.1 
6.5 
3.1 
3.6 


7.0 
9.8 
6.C 
6.8 
5.2 
10.0 
8.4 


Grade  8 

R2  14  3.6  3.0  5.5 

R3  9  0.9  0.3  2.8 

R4  10  0.6  1.7  2.3 

R5  14  2.3  1.8  2.3 

R6  18  5.1  4. ft  7.0 

R7  19  3.5  4. J  3.9 

R8  Iv.  2.5  2.3  3.9 


Grade  12 

R2  14  3.9  3.0  6.1 

R3  11  4.2  4.7  6.3 

R4  19  0.9  1.6  3.1 

R5  14  1.4  1.4  7.0 

R6  15  3.4  4.1  5.7 

R7  19  2.4  3.0  3.4 

R8  18  1.5  3.7  5.9 


*  Blocks  R2,  R5,  and  R7  are  identical  for  grades  8  and  12.    Except  in 
these  cases,  identity  of  block  numbers  across  grades  does  not  icply  that 
blocks  are  identical. 
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Table  10-6 


Average  Item  Percent  Correct 
for  Each  Block  Position 

Number  of           Position                Position  Position 
Block*             Items  111 

Grade  4 

R2                       14                74.5                       72.9  69.9 

R3                          7                 72.0                        69.2  61.4 

R4                          7                 56.1                        57.7  52.7 

R5                        15                 75.3                        74./  73.5 

R6                        14                 69.2                        65.9  65.4 

R7                        15                 71.0                        69.6  67.5 

R8                        12                 72.7                        69.2  65.7 


Grade  8 

R2  14  58.8  57.2  53.0 

R3  9  85.7  84.9  81.4 

R4  10  73.5  75.7  68.7 

R5  14  45.9  47.0  47.6 

R6  18  67.3  65.3  64.9 

R7  19  71.3  67.1  65.8 

R8  16  84.7  83.6  79.4 


Grade  12 

R2  14  70.9  71.5  69.5 

R3  11  70.8  69.7  67.5 

R4  19  73.6  73.2  72.1 

R5  14  62.2  62.8  61.2 

R6  15  66.0  65.9  62.0 

R7  19  80.2  79.3  76.3 

R8  18  76.3  75.9  72.4 


*  Blocks  R2,  R^,  and  R7  are  identical  for  grades  8  and  12.    Except  in 
these  cases,  identity  of  block  numbers  across  grades  does  not  imply  that 
blocks  are  identical. 
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2.    Calibrate  the  items  and  investigate  differential  item  functioning 
across  grade  levels  and  racial/ethni    and  gender  groups.    Strictly  speaking, 
the  findings  on  reading  block  position  effects  imply  the  need  for  an  item 
re:  onse  model  that  allows  item  parameters  to  depend  on  block  position;  that 
is,  a  model  in  which  each  item  in  the  main  assessment  would  have  three  sets  f 
item  parameters,  one  corresponding  to  each  possible  position.  Such  a  model 
would,  of  course,  be  unwieldy  and  its  parameters  could  not  be  estimated 
accurately  wit\.  current  sample  sizes.      Fortunately,  however,  the  focused-BIB 
portion  of  the  assessment  has  the  property  that  each  block  appears  in  each 
position  exactly  once.    Therefore,  item  parameter  estimates  based  cn  the 
focused-BIB  assessment  can  be  viewed  as  appropriately  weighted  averages  of 
these  three  sets  of  parameter  estimates.    This  property  of  the  item  parameter 
estimates  would  not  hold  if  the  three  intercorrelation  booklets  were  included 
in  the  item  calibration  (see  Tables  4-2,  4-3,  and  4-4);  hence,  the  calibration 
was  based  only  on  the  focused-BIB  portion  of  the  assessment. 

For  all  three  grade/ages  combined,  th^  BILOG  prog,  ani  (Mislevy  &  Bock, 
1982)  was  used  to  obtain  item  parameter  estima^.^s  cn  a  provisional  scale, 
based  on  the  three- parameter  logistic  model.     Sampling  weights  were  not  used 
in  this  phase  of  the  analysis.     Parameters  were  estimated  even  for  previously 
administered  items;  that  is,  parameter  values  for  old  items  were  rot  assumed 
equal  to  their  previous  values.    To  reduce  costs,  a  systematic  sample 
consisting  of  half  of  the  students  in  the  focused-^. B  samples  9[Main-Rdg], 
13[Main-Rdg] ,  and  17[Main-Rdg]  was  used  in  the  item  calibration.      The  three 
samples  were  treated  as  distinct  subpopulations  in  the  BILOG  run;  that  is,  5  i 
estimating  the  item  parameters,  the  densities  for  the  three  grade/age  groups 
were  not  assumed  to  be  tha  same. 

Using  the  method  described  in  Chapter  7,  a  graphical  analysis  was 
conducted  to  determine  whether  it  was  reasonable  to  assume  common  item 
response  functions  for  all  three  age  classes.    For  each  item  and  each  group, 
expected  proper^  ic      correct  (see  Mislevy  6c  Sheehan,  p,  302)  for  each  of 
approximately  eight  proficiency  levels  were  obtaj»ied.    The  depaitures  of  these 
proportions  from  the  common  estimated  item  response  func*:ion  were  examined. 
In  the  case  ojT  four  items ^  the  assumption  of  a  common  item  response  function 
for  all  age  classes  appeared  to  be  seLiously  violated.  Therefore,  a  second 
calibration  was  conducted  in  which  the^e  items  were  allowed  to  have  different 
item  response  functions  across  age  classes.     Table  10-7  lists  these  items  and 
indicates  how  they  were  treated  in  the  second  calibration.    The  first  two 
items  refer  to  a  single  passage  about  getting  summer  Jobs.     It  is  not 
surprising  that  these  items  functioned  differently  for  students  of  different 
ages:     Given  an  eighth  grader  and  a  twelfth  grader  with  equal  reading 
proficiency,  the  older  student  is  more  likely  to  answer  correctly  because  of 
greater  familiarity  with  Jiethods  for  getting  summer  jobs.    The  next  two  items 
in  Table  10-7  required  treatment  as  a  separate  item  for  grade  4.    As  noted 
earlier,  the  response  mode  for  grade  4  differed  from  that  for  grades  8  and  Iz 
(Table  10-2).  It  is  unclear,  though,  why  these  items  in  particular  should  be 
affected.    Perhaps  certain  items  are  more  susceptible  to  response  mode 
dif ference<; .    Using  the  same  method,  residuals  were  also  examined  for  Black, 
Hispanic,  and  White  students  and  for  males  and  female.;.    No  major  or 
systematic  departures  were  found 
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Table  10-7 

Items  for  Which  Multiple  Item  Response  Functions  Were  Estia[ia',:ed 


NAEP  Item 


Grade /Ape 


Calibration  Procedure 


NO032O2 

Summer  job-best  time  to  find  8/13,  12/17  Treat  separately 


N003204 

Summer  job-references 

N002401 

Mosquito -size  exaggerated 


8/13,  12/17  Treat  separately 

4/9,  8/13  Treat  separately 


R000904 

Naomi  James-year  she  set  record     4/9,  8/13,  12/17    Tr  at  4/9  separately; 

treat  8/13  and  12/17 
together 


Item  parameter  estimates  for  the  215  items  administe.  ^cl  to  the  focused- 
BIB  and  intercorre .ation  samples  are  given  in  Table  F-1  of  appendix  F. 


3.    Obtain  the  Proficiency  means  and  standard  errc  s  on  a  provisional 
scBuB  for  the  NAEF  reporting  groups  for  the  focused-BIB  ar.d  intercorrelation 
samples,    hs  i.i  198^,  responses  to  reading  items  were  summarized  in  a  single 
reading  scale.    Although  Male  Vc^lues  were  obtained  for  the  grade/age  samples, 
only  the  grade  subsamples  of  the  focused-BIB  reading  sample  were  used  as  the 
basis  of  the  reading  cross-srctional  report;  sample  sizes  were  4,534,  4,404, 
and  4,250  for  grades  4,  8,  and  12.      For  each  of  the  NAEP  reporting 
categories,  means  and  standard  eri'ors  were  computed  using  trhe  technology 
described  in  Chapter  9.    Note  that  leading  background  itemj  were  used  as 
conditioning  variables  for  the  focused-BIB  samples.    However,  with  the 
exception  of  a  single  item,  these  items  were  not  administered  to  the  students 
in  the  intercorrelation  samples,  who  received  civics  and  history  background 
items  (with  one  reading  background  item)  instead.    Because  different 
conditioning  modelr  were  required,  the  conditioning  procf,ss  was  conducted 
separatsly  for  the  focused-BIB  and  intercorrelation  samples .    For  each  of 
t^ese  samples,  a  list  of  the  conditioning  variables,  the  scheme  for  coding 
them,  and  their  estimated  effects,  appear  in  Tables  C-12  to  C-17  or 
Appendix  C, 


4,    Determine  the  appropriate  metric  for  reporting,  investigate  linkage 
of  the  1988  main  N^iEP  results  with  the  198^  reading  scale.    Another  component 
of  the  analysis  was  the  exploration  of  the  feasibility  of  linking  the  1988 
main  NAEP  results  with  the  reading  trend  scale  established  in  198^'.     Such  a 
linkage  appeared  unlikely  to  succeed  because  the  1988  assessment  differed  in 
major  ways  from  the  1984  assessment.    The  time  of  year  at  which  assessrrents 
were  administered,  the  age  definition  for  17 -year -olds,  and  the  response  mode 
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(see  Table  10-2)  all  differed.    Furthermore,  although  72  of  the  items 
administered  in  1988  were  also  given  in  1984,  these  items  did  not  appear 
within  intact  1984  item  blocks.    Through  our  investigation  of  the  1986  reading 
anomaly,  we  learned  that  reading  items  may  behave    n  a  substantially  different 
manner  when  they  appear  in  altered  contexts  and  that  other  s«eming]y  minor 
changes  can  have  major  effects  on  equating.    Nevertheless,  three  equating 
approaches  were  applied  in  an  attempt  to  achieve  an  approximate  linking.  If 
the  three  equating  efforts  were  to  yield  similar  equating  functions,  this 
would  support  the  validity  of  the  link. 

Two  of  the  equating  efforts  took  advantage  of  the  existence,  at  ages  9 
and  17,  of  subsamples  of  the  bridge  to  1984  and  the  1988  main  NAEP  samples 
that  had  common  age  definitions  and  times  of  assessment.    At  each  of  these  age 
levels,  the  RESOLVE  program  (which  implements  the  procedures  described  in 
Mislevy,  1984)  was  used  to  estimate  the  reading  proficie-^cy  distribution  of 
the  bridge  and  main  NAEP  students,  based  on  item  responses  and  item 
parameters.     (The  1984  item  parameters  were  used  for  the  bridge  to  1984,  item 
parameters  from  step  2,  above,  were  used  for  the  rain  NAEP  sample.)  The 
linear  function  required  to  match  the  means  and  standard  deviations  of  the 
estimated  bridge  and  main  NAEP  distributions  was  then  obtained.     In  addition 
to  these  two  common  pop^lo.-ion  equating  methods,  a  common  item  equating 
procedure  was  applied  to  items  common  to  1984  and  1988  despite  the  face  that 
items  did  not  appear  in  intact  blocks.      The  S*:ocking-Lord  (1983)  procedure, 
implemented  in  the  TBLT  program  (Stocking,  1986;  see  also  Sheehan,  1988)  was 
used  to  find  the  best- fitting  linear  transformation  for  mapping  the 
provisional  pai:ameters  for  the  68  common  items,  ol'tained  in  the  1988 
calibration,  to  the  existing  1984  parameters  for  these  items.  (The  three  items 
common  to  1984  that  were  treated  separately  across  age  classes  in  1988  were 
not  included  in  the  set  of  items  common  between  l^^T"  and  1984.     In  addition 
one  1988  open-ended  it^m  that  had  been  adminif.tered  in  1984  was  not  treated  as 
a  common  item  because  a  f.tady  of  score  drift  [Zwick,  1988]  showed  that  rating 
criteria  were  not  cxppliso  consistently  in  1984  and  1988.    The  resulting  number 
of  effective  common  items  was  68. > 

The  estirctceu  egaating  functions  from  thes'^  three  methods  were  quite 
disparate,  reinfor'-ing  the  preliminary  conclusion  that  the  two  assessments 
were  too  different  to  be  linked  in  a  satisfactory  way.    Therefore,  as  in  1984, 
the  mean  and  standard  deviation  for  the  three  grade/ages  combined  were  set  to 
250.5  and  50.^    We  plan  to  link  the  1990  main  NAEP  results  to  this  new  scale. 

After  determining  the  metric  for  reporting,  another  decision  that  had  to 
be  made  was  whether  to  combine  the  results  from  the  innercorrelation  samples 
with  those  from  the  ^ocused-BIB  samples.    As  noted  earlier,  only  the  focused 
BIB  samples  were  used  in  item  «     ibration,  but  scale  values  were  obtained  for 
both  samples.    However,  the  results  from  the  intercorrelation  samples  differed 


^Tht2  metric  in  which  the  item  parameters  iii  Appendix  F  are  given  differs 
by  a  linear  tr   '^fo^mation  from  the  reading  scale  used  for  reporting;  Lettii.g 
6  represent  the    roticiency  metric  that  corresponds  to  *:he  i^em  parameters  and 
letting  RS  represent  the  metric  of  the  reporting  scale,  the  required 
transforraation  is  RS  -  508  +  250.5. 
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in  two  ways  from  the  focused- BIB  results.    First,  the>  were  not  balanced  v^ith 
respect  to  block  position  and  second,  they  were  based  on  a  less  complete 
conditioning  model .     Examination  of  subgroup  means  and  standard  deviations  f 
the  two  samples  showed  some  small  but  systematic  differences  between  the  two. 
In  particular,  the  subgroup  means  for  sample  9[Main-Int]  tended  to  be  slightly 
higher  than  those  for  sample  9[Main-Rdg]  aud  the  standard  deviations  for 
sample  13[Main-Int)  tended  to  be  lower  than  those  for  13 [Main-Rdg] . 
Therefore,  only  the  focused- BIB  results  were  used  as  the  basis  for  Learning  to 
Read  in  Ovr  Nation's  Schools:    Instruction  and  Achievement  in  1988  at  Grades 
4,  8,  and  12  (Langer,  Applebee,  Mullis,  &  Foertsch,  1990).    The  scale  values 
for  the  intercorr elation  samples  are  available  on  the  1988  public-use  data 
tapes . 

5.    Conduct  a  study  of  the  relation  between  the  reading  proficiency  ox 
fourth  graders  and  teacher  background  asid  instructional  variables.  Teacher 
background  data  and  .information  on  instructional  methods  used  with  specific 
students  were  collected  from  the  reading  teachers  of  3,901  fourth  graders 
(about  86  percent  of  the  fourth-grade  students  in  sample  9 [Hain-Rdg] ) .  There 
are  several  possible  reasons  for  the  absence  of  teacher  data  for  1^  percent  of 
the  students:     Sc  Tie  teachers  may  not  have  been  identified,  others  were  not 
sampled,  and  others  may  have  failed  to  complete  questionnaires.  Also, 
teachers  were  not  required  to  complete  questionnaires  for  more  than  ten 
students.    Further  detail  about  thj  .process  for  sampling  teachers  is  given  in 
Chapters  3  and  8. 

For  reasons  explained  here,  the  analysis  of  the  relation  of  these 
teacher  data  to  reading  proficiency  requires  the  use  of  separate  set:,  of 
reading  scale  values  and  sample  weights.    The  goal  of  these  analyses  was  to 
report  reading  proficiency  results  for  each  possible  response  to  selected 
teacher  items.     For  example,  student  reading  proficiencies  were  computed  for 
each  of  several  levels  of  teacher  education  and  for  each  of  several  levels  of 
frequency  of  individualized  instruction.     In  order  to  avoid  biases  in  these 
analyses  (see  Chapter  9) ,  it  was  necessary  to  include  as  conditioning 
variabler  the  teacher  items  that  had  been  selected  for  reporting.  These 
conditioning  variables  were,  of  course,  unavailable  for  the  nonrandom  14 
percent  of  the  focused-BIB  students  whose  teachers  did  not  complete 
questionnaires.    Because  the  missing  data  problem  affected  a  large,  nonrandom 
portion  of  the  sample,  the  use  of  a  common  conditioning  -!iodel  in  which  the 
teacher  items  were  s^.mply  treated  as  missing  for  certain  stodents  could 
produce  a  substantial  v^iolation  of  the  assuiption  of  equal  residual  variances 
in  each  conditioning  cell.    Therefore,  a  separate  set  of  plausible  values  was 
obtained  for"  students  with  teacher  data  and  tiese  wero  compared  to  the 
original  set.     Because  there  were  some  slight  differences,  the  two  sets  of 
plausible  values  were  retained — one  to  be  used  for  analyzing  the  relation  ^f 
reading  proficiency  to  the  teacher  data  and  one  for  other  analyses  of  teading 
data.    I  c  the  teacher-based  plausible  values,  a  list  of  the  conditioning 
variables,  the  scheme  for  coding  them,  and  their  estimated  effects,  appear  in 
Table  C-18  of  Appendix  C. 

In  any  case,  analyses  'nvolving  the  teacher  data  must  be  conducted 
separately  from  other  analyses  because  of  the  need  to  use  a  separate  set  of 
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sampling  weightr .    Fol  the  teacher  analyses,  the  teacher-based  student  weights 
must  be  used.    Ine  teacher-student  weight  is  the  nonresponse- adjusted  student 
weight  further  adjusted  for  the  teacher's  probability  of  selection  and  for 
teacher  nonresponse.     (Some  further  adjustments  were  also  applied;  see  Ch  ter 
8  and  The  1988  National  Assessment  of  Educational  Progress — Sampling  and 
Weighting  Procedures ,  Final  Report  [Rust,  Bethel,  Burke,  &  Hansen,  1990].) 
The  teacher-based  plausible  values  and  the  teacher-student  weights  are 
available  on  the  1988  public-use  data  tapes. 

Analyses  involving  the  teacher  background  and  instructional  variables 
are  included  in  Learning  to  Read  in  Our  Nation's  Schools:    Instruction  and 
Achievement  in  1988  at  Grades  4,  8,  and  12  (Langer,  Applebee ,  Mullis,  & 
Foertsch,  1990). 
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Chapter  11 

DATA  ANALYSIS  FOR  THE  WRITING  ASSESSMENT^ 


Eugene  G.  Johnson 


Educational  Testing  Service 


This  chapter  describes  the  analyses  carried  out  on  the  responses  to  che 
writing  tasks  and  the  background  items  in  the  198^  and  1988  assessments  of 
writing  that  led  to  the  results  reported  in  The  Writing  Report  Card,  198^^-88: 
Findings  from  the  Nation's  Report  Cdrd  (Applebee,  Langer,  Mullis,  &  Jenkins, 
1990)  and  Learning  to  Write  in  Our  Nation's  Schools:  Instruction  and 
Achievement  in  1988  at  Grades  4,  8,  and  12  (Applebee,  Langer,  Mullis,  Jenkins, 
&  Foertsch,  1990).    The  emphasis  is  on  the  methods  and  results  of  the 
procedures  used  to  develop  the  average  response  method  scale  scores  that 
formed  the  basis  of  those  reports. 

The  analysis  of  the  1988  writing  data  consisted  of  twc  components.  The 
first  component  was  designed  to  measure  trends  in  writi.ig  achievement  since 
1984.    Trends  in  writing  achievemeat  were  measured  by  comparing  the  r<?sponses 
to  a  set  of  writing  tasks  by  students  assessed  in  1984  wit'i  the  responses  to 
the  same  set  of  writing  tasks  for  students  assessed  in  198. L    The  major 
analyses  were  made  for  trends  in  average  task  accomplishment  (primary  trait) , 
based  on  a  writing  scale  developed  using  ^he  average  response  method,  although 
trends  in  writing  mechanics  and  trends  in  overall  writing  fluenc^'  (based  on 
holistic  scoring)  were  o^so  measured.    The  data  forming  the  bas      for  these 
analyses  were  from  the  1988  hrMge- to-1984  (trend^  samples  and  c      1984  main 
sa^Aples  (these  samples  are  c        ^d  below).    The  techniques  used  to  measure 
trends  in  writing  achievemei  discussed  in  section  11.1. 

The  second  component  in  the  analyses  of  the  writing  data  was  desig^.ed  to 
analyze  the  responses  to  the  writing  tasks  in  the  main  assessment,  to  develop 
a  cross -sectional  writing  scal^  based  on  these  data,  to  examine  the 
association  between  wricing  ability  and  writing  process  and  instruction,  and 
to  determine  the  effect  that  time  allocated  has  on  writing  ability.  These 
analyses  were  based  on  the  data  from  the  1988  main  assessment  samples. 
Secti  n  11.2  provides  a  description  of  the  analyses  of  the  cross-sectional 
data. 


^The  statistical  programining  for  the  average  response  method  was  ably 
performed  by  Bruce  Kaplan      Data  analysis  and  additional  statistical 
programming  were  performed  by  Bruce  Kaplan,  Jo -Ling  Liang,  Mike  Narcowich,  and 
Inge  Novatkoski.    The  author  is  indebted  to  Mary  Varone  for  typing  the 
manuscript. 
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rhe  specific  samples  used  for  the  analysis  of  writing  achievement  ir 
1984  and  1988  are  (by  age  cohort): 


Time  ( 

Df 

Modal 

Sample  Code 

Sample  Type 

Assessment 

Age  Definition 

Grade 

9:84 

Main 
ncii.li 

Calendar  year 

4 

9[Main-Wrt] 

1988 

Main 

Winter, 

spring 

Calendar  year 

4 

9[Main-LWr] 

1988 

long  Writing 

Winter, 

spring 

Calendar  year 

4 

9[Br84-RW] 

1988 

Bridge  to  1984 

Winter 

Calendar  year 

4 

13:84 

1984 

Main 

Fall 

Calendar  year 

8 

13[Main-Wrt] 

1988 

Main 

Winter, 

spring 

Calendar  year 

8 

13[Main-LWr] 

1988 

Long  Writing 

Winter, 

spring 

Calendar  year 

8 

13[Br84-RW] 

1983 

Bridge  to  1984 

Fall 

Calendar  year 

8 

17:84 

1984 

Main 

Spring 

Oct.  -  Sept. 

11 

17lMain-Wrtl 

1988 

Main 

Winter, 

spring 

Calendar  year 

12 

17(Main-LWr] 

1988 

Long  Writing 

Winter, 

spring 

Calendar  year 

12 

17lBr84-RW] 

1988 

Bridge  to  19 

Spring 

Oct.  -  Sepr. 

11 

11.1    MEASUREMENT  OF  TRENDS  IN  WRITING  ACHIEVEMENT 

The  data  for  the  1988  point  in  the  eiwalysis  of  trcjnds  in  wricing  comes 
from  the  1988  bridge- to-1984  samples,  which  provide  a  bridge  to  the  1984 
assessment  and  match  ::hat  assessment  in  terms  of  the  time  of  administration 
and  age  definitions.     The  data  for  the  1984  point  in  writing  trend  comes  from 
the  responses  of  students  within  the  1984  main  (i.e.,  BIB)  assessment.  All 
analyses  of  trends  in  writing  performance  were  based  on  grade-eligible 
students  only.    For  reasons  given  below,  both  the  1988  .  nd  the  1984  points 
were  determined  by  scores  provided  by  raters  scoring  'Jhe  papers  in  1^88. 

The  items  on  which  the  trends  in  writing  achievement  are  based  are  showm 
in  Table  11-1.    The  table  shows  the  block  that  contained  the  item  in  1984  and 
the  trend  booklets  containir';  the  item  in  1988.     A  total  of  twelve  writing 
tasks  were  used  to  measure  u/ends  in  writing  achievement,  with  six  tasks 
presented  at  each  grade  in  1984  and  1988.    To  allow  comparisons  in  writing 
ability  across  grades,  three  of  the  six  tasks  presented  to  the  fourth- grade 
students  in  1988  were  also  presented  to  the  eighth-grade  students;  three  of 
the  eighth-grade  tasks  were  also  presented  to  the  eleventh- grade  students;  and 
one  task  was  presented  at  all  three  grades. 


11.1.1  Primary  Trait  Scoring  of  the  Writing  Tasks  &nl  Measures  of  Scorer 

Effect 

All  writing  exercises  from  the  1988  assessment  were  scored  for  task 
accomplishment  (primary  trait).    For  the  purposes  of  analysis,  the  student 
responses  wer*,  coded  as  0-not  rated,  1-unsatisf actory ,  2-minimal,  3«adequate, 
and  4-elabora\:ed.    A  20  percent  random  subsample  of  all  the  papers  scored  were 
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Table  11-1 


Assignment  of  1984-1988  Writing  Trend  Items  in  1984  and  1988 


Writing  Tesk 

N0003  Recreation  0pp. 
N0004  Food  On  Frontier 
N0005  Dissecting  Frogs 
N0006  XYZ  Company 
N0009  Radio  Station 
NOOlO  Appleby  House 
N0O76  Flashlight 
N0147  Plants 
N0148  Spaceship 
N0180  Space  Program 
N0190  Job  Application 
NO 210  Bike  Lane 


1984 

BIB-spiral  Blocks 
  Grades   


E 
G 
G 
V 
C 
E 


8 

C 
D 
E 
E 
G 
G 
V 


11 

C 
D 


G 

V  * 


E 
E 
G 


1988 

Bridge  to  1984  Booklets 
■   Grades   


52  54 
54,55 
54,55 
56 

51,53 
52,54 


8 

52,54 
51,54 
53,55 
53,55 
55,56 
55,56 


ii 

52,54 
51,54 


55,56 


53,55 
53,55 
55,56 


*  Block  V  never  appeared  with  any  other  writing  block  in  1984  (all  other 
blocks  appeared  with  every  other  blocK  at  the  same  grade  in  198^) . 
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re^wed  by  a  second  rater  to  provide  an  estimate  of  interrater  reliability 
Tablo  11-2  shows  scorer  reliability  for  each  essay  as  meas'ired  by  the 
intraciass  correlation.    Additionally,  Table  11-2  shows  the  percentage  of 
exact  score  agreement  between  the  firs,  and  second  raters.    The  reliabilities 
and  percents  of  exact  agreement  are  generally  high  and  are  consistent  with 
equivalent  values  from  prior  assessments. 

Although  the  measures  of  scorer  agreement  in  NAEP  have  been  consistently 
high,  we  recognized  the  possibility  that  there  might  be  variation  between  the" 
ratings  provided  by  the  group  of  scorers  assembled  in  1988  and  the  scorers 
assembled  in  1984.     If  present,  this  variation  would  add  a  confounding  effect 
in  ths  measurement  of  trend.    The  most  direct  way  of  controlling  the  effect  of 
across-year  variation  in  scoring  would  be  to  eliminate  it  entirely  by 
rescoring  all  of  the  1984  data,  using  the  same  set  of  scorers  who  scored  the 
1988  data.    Unfortunately,  resources  did  not  allow  for  tne  rescoring  of  the 
fi  LrV^.^^^^  writing  papers  but  did  allow  for  a  rescoring  of  approximately 
il.OOO  of  the  papers  given  in  1984.    The  rescored  papers  for  a  given  item 
constituted  approximately  a  25  percent  sample  of  all  1984  papers  and  consisted 
of  all  grade -eligible  respondents  to  two  or  three  of  the  1984  booklets 
containing  that  item. 

The  hope  was  that  the  between-year  variability  in  scoring  would  be  low 
enough  to  permit  the  use  of  the  full  set  of  the  1984  data.     Tables  11-3  and 
li-4  show  the  results  of  the  comparison  of  the  rescore  of  the  1984  data  with 
the  scores  assigned  to  the  papers  in  1984.    Table  11-3  shows,  by  grade  and 
item,  the  average  difference  between  the  1988  rescore  and  the  1984  score  and 
the    standard  deviation  of  the  difference.    Table  11-4  shows  the  distribution 
of  the  difference  between  the  rescore  and  the  original  score,  again  by  age  and 
item.    The  average  difference  between  the  rescore  and  the  original  score  is 
-.019  for  grade  4,  -.046  for  grade  8  and  -.056  for  grade  11,     differences  of 
the  same  general  magnitude  as  the  between-group  differences  that  were  reported 
in  the  previous  Writing  Report  Card.    Although  the  1988  scorers  app-a-  to  have 
been  more  strin^.v-nt  on    average  than  the  1984  scorers,  the  tables  show  that 
the  difference  between  the  two  groups  of  scorers  is  not  simply  a  consistent 
drift. 


In  light  of  the  diffeiences  between  the  1984  and  the  1988  scoring 
direct  comparisons  between  the  1988  results  and  the  original  1984  results  were 
not  considered  acceptable.     Const   lently,  the  1984  trend  point  was  based  on 
the  rescored  data  only.    The  resultant  sample  sizes  for  the  trend  report 
analyses  are  given  in  Table  11-5. 

(It  should  be  noted  that  another  option  that  was  considered  was  to 
adjust  the  original  1984  scores  to  reflect  the  observed  relationship  between 
the  rescores  and  the  original  scores  on  the  set  of  rescored  data.     Such  an 
adjustment  of  the  1984  scores  would  ise  a  multiple  imputation  procedure  in 
which  ad^-'ted  1984  scores  for  a  given  item  are  drawn  from  the  multinomial 
distribution  of  possible  original  scores  for  that  item  conditional  on  a  given 
rescore  value.    While  such  an  approach  would  use  the  full  set  of  1984  data, 
the  results  would  include  a  component  of  uncertainty  inherent  in  the 
imputation  of  scores.     This  component  can  be  Inrge:     U mined  analyses  indicate 
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Table  11-2 


Percentages  of  Exact  Score  Aj^'eement  and  Interrater  Reliability 
for  the  Primary  Trait  Scoring  of  the  1988  Writing  Trend  Items 


Gra  la  4 


Grade  8 


Gradfc  11 


Percent 
Exact  Relia- 


Percent  Percent 
Exact       Relia-         Exact  Relia- 


NAEP^tem 

N00O602 

XYZ  Company 

N00O902 

Radio  Station 

N001002 

Appleby  House 

N007602 
Flashlight 

N014702 
Plants 

N014802 
Spaceship 

N00O302 

Recreation  0pp. 

N0nO402 

Food  on  Frontier 

N00O502 

Dissecting  Frogs 

N018002 

Space  Program 

N019002 

Job  Application 

N021002 
Bike  Lane 

AVERAGE 


Agreement  bilitv       Agreement    bility      Agreement    bill tv 
97.1        .985  93.5  .920 


93.5  .950 

90.3  .916 

87.5  .874 

94.3  .946 

91.8  .952 


87.0  .886 


75.3 


.688 


85.4  .820 


79.9  .683 


76.1  .642 


89.3 


89.9 
92.3 


.888 


90.8  .931 


93.1  .863 


.928 
.922 


84.9  .866 


92.4  .937 


83.0  .773 


90.0  .900 
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Table  11-3 


Mean  and  Standard  Deviation  of  (Rescore  -  Original) 
for  the  20X  Reuore  of  1984  Writing  Responses 


Grade  A  Grade  8  Grade  11 

NAEP  Item  Mean       S.  D.  Mean       S.  D.  Mean       S,  p, 

N000602 

yaz  Company  -.010      .311  .057  .559 

N000902 

Radio  Station  -.039      .342  -.034  .450 

N001002 

Appleby  Hoyse  .003      .339  -.055      .538  -.064  .375 

N007602 

Flashlight  -.043  .367 

N014702 

Plants  .000  .356 

N014802 

Spaceship  -.021  .399 

N000302 

Recreation  0pp.  -.046      .588  -.118  .547 

N000402 

Food  on  Frontier  -.117      .533  -.092  .454 

N000502 

Dissecting  Frogs  -.063  .563 

N018002 

Space  Program  -.086  .517 

N019002 

Job  Application  .031  .504 

N021002 

Bike  Lane  --006  .527 


OVERALL  -.019  -.046  -.056 
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Table  11-4 


Distribution  of  (Rescore-Original,  , 
for  the  20X  Rescore  of  1984  Writing  Responses 
Percent  of  Responses  Where  (Restore -Original)  Equals  -1,0  or  1 


Grade  4  Grade  8  Grade  11 

Rescore-Original    Rescore-Original  Rescore-Original 


NAEP  Item 


N000602 

XYZ  Company  2.6    93.7    2.6         2.5    83.0  9.7 

N000902 

Radio  Station  5.9    90.8    3.^       10.5    81.4  7.5 

N001002 

Ap?leby  House  5.3    89.5    4.9       15.8    73.1    10.3         9.5    86.4  3.8 

N007602 

Flashlight  8.2    87.5  3.9 

N014702 

Plants  4.8    90.2  4.1 

H014802 

Spaceship  6.0    87.7  r.3 

N0OO302 

Recreation  0pp.  11.9    75.0    10.1       15.5    75.4  6.9 

N000^02 

Food  on  Frontier  7.6.2    75.0      1.1       13.3    81.2  4.6 

N000S02 

Dissecting  Frogs  17.0    70.1  12.2 


N018002 

Space  Program 

N019002 

Job  Application 

N021002 
Bike  Lane 


14.7    77.4  6.-' 
6.4    83.7  7.6 
il.6    75.7  11.*^ 


OVERALL  5.4    89.9    4.0       12.3    76.2      9.5       11.8    80.0  0.9 
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Table  11-5 


Sample  Sizes  for  Primary  Trait  Analyses 
of  Trends  in  Writing  Performance 


NAEP  Item 


Grade  4 


1988  1984 
^  ridge  Rescore 


Grade  8 


1988  1984 
Bridge  Rescore 


Grade  11 


1988  1984 
Bridge  Rescore 


N000602 

XYZ  Company 


1152 


544 


1334 


N000902 

Radio  Station 


1234 


585 


1364 


612 


N001002 

Appleby  House 


925 


530 


1256 


588 


1041 


599 


N007602 
flashlight 


614 


C09 


N014702 
Plants 


1285 


656 


NO 1480 2 
Spaceship 


1258 


611 


N000302 

Recreation  0pp. 


1372 


494 


1242 


521 


N000402 

Food  on  Frontier 


1339 


603 


1212 


629 


N000502 

Dissecting  Frogs 


1356 


641 


N018002 

Space  Program 


1195 


632 


NO19002 

Job  Application 


1169 


603 


N021002 
Bike  Lane 


11783 


636 
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that  the  standard  airors  based  on  the  rescored  data  only  are  of  the  same 
general  size  as  the  standard  errors  of  the  full,  imputed-up-to-1988  data.) 

11. 1.2  The  Writing  Trend  Scale  Based  on  the  Average  Response  Method 

The  major  analysis  of  trends  in  wricing  achievement  was  based  on  the 
average  response  method  (ARM)  of  scaling  nonbinary  data  (Beaton  and  Johnson, 
1987,  1990),  although  analyses  of  individual  items  were  conducted  also. 
Since  the  measurement  of  trend  was  based  on  the  rescored  data,  the  ARM  writing 
scale  established  in  1984  could  not  be  used.     Instead,  a  new  ARM  writing  scale 
was  constructed  based  on  the  trend  items  given  in  1980.    Because  certain  of 
the  between- item  correlations  are  not  estimable,  the  ARM  technology  has  been 
generalized  to  allow  for  the  imputation  of  missi.;^  correlations. 

11.1.2.1       Overview  of  the  Average  Response  Method 

The  avtrage  response  method  begins  with  a  defined  composite  of  the 
(primary  trait)  scores  for  a  set  of  p  exercises  and  provides,  for  each 
assessed  student,  draws  from  the  distribution  of  potential  values  for  that 
composite.     If  a  student  had  responded  to  all  the  exercises  going  into  the 
composite,  then  that  student's  ARM  score  would  be  directly  calculable,  without 
error,  by 

6  -  a'x 

where  x  is  the  vector  of  the  subject's  scores  on  the  p  questions  in  the 
composite  and  a  is  a  vector  of  p  arbitrary  constants,  which  for  the  ARM 
writing  scale  are  each  equal  to  1/p  (since  the  ARM  writing  scale  is  defined  as 
the  predicted  average  performance  across  the  set  of  p  writing  questions). 

Because  each  respondent  is  presented  only  a  subset  of  the  questions,  the 
respondent's  compcsite  value  is  unknown  and  so  must  be  estimated.    Such  an 
estimate  is  provided  by  the  ARM  technology.    Briefly,  the  ARM  technology  is  a 
kind  of  multiple  regression  that  produces  for  each  student  a  s^t  of  plausible 
values,  each  of  which  predicts  what  that  student's  composite  score  might 
plausibly  be,  based  on  the  student's  scores  on  the  exercises  in  the  composite 
that  were  presetted  to  the  student  and  based  on  the  student's  status  on  a 
selected  set  of  background  variables,  called  the  conditioning  variable.^.  (The 
purpose  of  the  conditioning  variables  is  to  improve  the  prediction  of  subgroup 
effects  given  the  sparse  quantity  of  information  available  for  any  individual. 
Their  exclusion  vould  lead  to  biased  estimates  of  subgroup  effects.  See 
Chapter  9  for  a  general  definition  and  description  of  plausible  values  and 
conditioning  variables) . 

Let  Xi  represent  the  (row)  vector  of  responses  of  the  ith  student  to  the 
questions  in  the  ARM  composite  that  were  presented  to  that  student  and  let  y^ 
be  the  (row)  vector  of  values  of  that  student's  conditioning  variables.  Then 
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a  plausible  value  from  the  conditional  distribution  of  $  given  the  observed 
data  Xi  and       for  student  i  is 


^'ik  -  ^iP  +  Vi^  +  Xiaj,  +  yi7j,  +  (11.1) 

where 

is  the  kth  plausible  value  of  the  ARM  composite 

P  is  the  (column)  vector  giving  the  change  in  the  composite 

for  unit  change  in  the  scores  on  each  of  the  questions  in 

A 

r  is  the  (column)  vector  of  effects  for  the  conditioning 

variables 

and  are  random  draws  from  a  multivariate  normal  distribution 

with  mean  vector  0  and  variance-covariance  matrix  S  where  S 
is  the^variance-covariance  matrix  of  the  parameter  estimates 
jp  and  r.     (ajj  and^y^  reflect  the  uncertainty  due  to  using 
sample  estimates  p  and  T  in  the  regression  equation.) 

€^  is  an  estimated  residual  drawn  from  a  normal  distribution 

with  mean  0  and  variance  a|  where  a|  is  the  variance  of  the 
predictive  distribution  of  6  given  the  observed  values  of  x^ 
and  y^. 

All  parameters  in  equation  (11.1)  were  estimated  by  least- squares 
technology.    To  accomplish  this,  it  is  sufficient  to  obtain  estimates  of  the 
elements  of  the  population  sum  of  squares  and  cross  products  matrix  of  the 
conditioning  variables  and  the  writing  questions: 


an  estimate  of  V'V  - 


Y'Y  Y'X 
X'Y  X'X 


(11.2) 


In  the  above,  Y  is  a  N  X  q  matrix  containing  the  values  of  the  q 
conditioning  variables  for  each  of  the  N  students  in  the  population;  X  is  a 
N  X  p  matrix  containing  the  scores  of  the  N  students  in  the  population  on  the 
p  exercises;  and  V  -  [Y  X]  .    If  Y  and  X  were  known  for  all  students  in  the 
population,  C  would  be  trivially  equal  to  V'V.    However,  since  only  a  sample 
of  the  students  in  the  population  were  assessed  for  writing  and  since  each 
sampled  student  was  only  presented  a  few  writing  questions,  many  of  the 
elements  of  Y  and  X  are  unknown.    Accordingly,  V'V  must  be  estimated.  The 
procedures  used  to  determine  an  estimate  C  of  V'V  will  be  deferred  to  the  next 
subsection. 

Since  the  ARM  composite  is  the  mean  of  the  individual  questions,  the 
estimate  C  generates  a  complete  set  of  sufficient  statistics  (the  normal 
equfttJ.ons)  for  the  standard  least- squares  prediction  of  an  ARM  composite  value 
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given  conditioning  variable  characteristics  and  responses  to  any  subset  of 
writing  questions.    Define  the  N  element  column  vector  T  by 


T  -  Xa 

where  the  elements  of  T,  0^,  are  the  values  of  the  composite  for  each  student 
in  the  population.     The  exact  value  of  6^  will  not  be  known  unless  the  student 
i  was  administered  all  p  of  the  exorcises.    The  plausible  values,  ^^j.,  of 
equation  (11.1)  are  determined  by  operatioiio  on  the  matrix  Cq  where  Cq  is  the 
estimated  population  sura  of  squares  and  cross  product  matrix  of  the 
conditioning  variables,  the  writing  exercises  and  the  composite.     Cq  is 
generated  by  the  matrix  C  and  the  transformation  matrix 


H  - 


Iq  0  0 
0    Ip  a 


by 


Ce  -  H'CH 


Y'Y  Y'X  Y'T 
X'T  X'X  X'T 
T'Y    T'X  T'T 


The  matrix  Cq  can  be  used  to  estimate  a  value  of       for  student  i  as 
follows : 

Let  Xj^  consist  of  the  columns  of  X  corresponding  to  the  writing 
exercises  presented^to  student  i  and  let       -  [Y  X^].     The  least-squares 
estimates  of  p  and  F  in  equation  (11.1)  are 


A 

r 

A 

Y'Y  Y'Xi 
Xi'Y  Xi'Xi 


Y'T 
Xi'T 


(11.3) 


and  the  standard  least-squares  point  estimate  of  the  composite  score  for 
student  i  is 


Oi  -        +  y^r 


This  value  is  the  ucan  of  the  predictive  distribution  of  potential  Bs  for  the 
individual  and,  thus,  does  not  take  into  account  the  fact  that  any  other  value 
from  this  predictive  distribution  might  also  have  been  the  student's  score. 
By  including  the  terms  accounting  for  the  uncertainty  in  the  estimation  of  a 
student's  composite  score,  the  plausible  values  0^^  allo^  the  more  complete 
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representation  of  what  is  known  and  what  is  not  known  about  the  student's 
composite  scores.    The  terms  accounting  for  uncertainty  are  of  two  types: 


1)  c^j^,  accounting  for  variability  of  potential  scores  of  an 
individual  about  the  conditional  mean  (of  the  distribution  given 
y^  ani  Xj^)  and 

2)  and  7j^,  accounting  for  uncertainty  due  to  using  sample 
estimates  of  ^  and  f  in  the  regression  equation. 

is  a  random  draw  from  a  N(0,aj)  distribution,  where       is  the 
residual  mean-squared-error  for  the  i egression  defined  by  (11.3).    The  vector 


7k 


is  a  draw  from  a  multivariate  normal  distribution  with  mean  0  and  variance- 
covariance  matrix 


The  values  of       and  -y^  are  held  fixed  for  all  student,  with  the  same  pattern 
of  missing  data. 

A  further  discussion  of  the  generation  of  ARM  plausible  values,  given  an 
estimate  C  of  the  stun  of  squares  and  closs  product  matrix  V'V,  appears  in 
Beaton  and  Johnson  (1990) .    The  next  section  considers  the  estimation  of  V'V 
for  the  1988  and  1984  writing  trend  scale. 


11.1,2,2 


Estimation  of  V'V 


As  noted  in  tho  previx^us  subsection,  the  basis  for  the  estimation  of  a 
predicted  value  for  any  student  is  an  estimate  C  of  the  full  sums -of -squares - 
£ind-cross  products  matrix 


V'V  - 


Y'Y  Y'X 
X'Y  X'X 


(11.4) 


from  which  all  other  necessary  matrices  and  estimates  are  deri  2d.     For  the 
construction  of  the  NAEP  writing  trend  scale,  six  separate  estima^^s  of  the 
cross-product  matrix  were  created:    one  for  each  of  the  three  grades  for  each 
of  the  years  1984  and  1988.    The  elements  of  the  estimate  C  of  V'V  for  a 
particular  grade  and  year  fall  into  three  general  types: 


Type  1:         Elements  that  are  directly  estimable  from  the  available  data 
for  that  grade  and  year;  these  are  sums  of  squares  and 
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cross-products  involving  the  coiiiitioning  variables  and  the 
items  presented  to  that  grade  in  that  year. 

Type  2:         Elements  that  must  be  estimated  based  on  relationships 

observed  for  another  grade  in  the  same  year  or  for  the  same 
grade  in  the  other  year;  these  are  suras  of  squares  and 
cross-products  involving  items  and  pairs  of  items  not 
administered  to  the  target  grade  and  year  but  administered 
to  another  grade  or  year. 

Type  3:         Elements  requiring  the  imputation  of  between  item 

correlations;  these  are  cross-product  terms  involving  pairs 
of  items  that  have  never  been  presented  together  so  that  the 
between  item  correlation  is  not  estimable. 

The  complexity  of  the  estimation  of  these  three  types  of  elements  of  the 
matrix  C  increases  as  one  moves  down  the  list,  with  elements  of  type  1  being 
the  most  straightforward  to  estimate  and  elements  of  type  3  being  the  most 
complex.    We  will  consider  the  estimation  of  each  of  the  three  types  in  turn. 


li. 1.2,3       Estimation  of  Type  1  Elements  of  C 

The  most  directly  estimable  elements  of  the  matrix  C  for  a  given  grade 
and  year  are  those  involving  the  conditioning  variables  and  the  items 
presented  to  students  of  that  grade  who  were  assessed  in  the  specified  year. 

In  the  matrix  C,  the  conditioning  matrix  Y  contains  the  values  of  39 
conditioning  variables  measuring  each  student's  status  on  a  set  of 
demographic,  background,  and  attitude  questions  as  well  as  specific  questions 
related  to  the  student's  attitudes  to  and  experiences  in  writing.    A  list  of 
the  conditioning  variables  and  the  scheme  for  coding  them  appears  as  Table  C  5 
in  Appendix  C. 

The  values  of  the  conditioning  variables  are  known  for  all  students  and 
so  Y'Y  in  the  matrix  C  is  directly  obtained  by  taking  the  sura  of  squares  and 
cross-products  of  the  conditioning  variables  for  each  student,  weighting  these 
by  the  student's  sampling  weight  and  then  summing  across  all  students  of  the 
given  grade  and  assessment  year. 

Also  directly  estimable  are  suras  of  squares  and  certain  cross-products 
involving  the  writing  tasks  presented  .o  students  in  the  given  grade  in  the 
given  assessment  year.    Specifically,  if  the  item  Xj  was  presented  to  a  sample 
of  students  of  the  given  grade  in  the  given  year,  then  the  following  terms  are 
estimable: 

Y'Xi    the  cross-product  vector  between  the  conditioning  variables  and 
the  item,  and 

Xj'X-    the  sura-of-squares  for  the  item.     Furthermore,  if  another  item,  X2 
was  also  presented  to  the  grade  in  the  given  year  and  if  a  sample 
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of  students  responded  to  both       and  X2,  then  the  between- item 
cross-product  terms  Xi*X2  and  ^2^1         also  directly  estimable. 

Because  of  the  spiral  design,  each  item  is  presented  to  a  subsample  of 
the  full  sample  of  grade- eligible  students  assessed  for  writing  in  a  given 
year.     Consequently,  the  simple  procedure  used  to  estimate  the  elements  of  Y'Y 
is  inappropriate  for  estimating  the  elements  Y'X^,  X^X^  and  X^Xj.  However, 
these  elements  can  'je  consistently  estimated  in  the  following  manner. 

Because  of  the  spiral  design,  we  can  assume  that  the  set  of  students  who 
responded  to  the  itew  X^  is  a  representative  sample  of  the  population  of  all 
gra 'e-eligible  students  in  the  given  year.    Consequently,  the  appropriately 
weighted  sample  mean  X^,  and  the  weighted  sample  variance,  S^^,  based  on  the 
total  sample  of  students  of  the  grade  in  the  assessment  year  responding  to  the 
item,  are  consistent  and  unbiased  estimates  of  the  population  mean  and 
variance  for  the  item.    A  consistent  estimator  of  the  sura- of -squared  scores  in 
the  population  for  the  item  is 

where  Wjot  is  the  stun  of  weights  for  all  grade-eligible  students  assessed  for 
writing  in  the  given  year. 

A  consistent  estimate  of  the  cross-product  element  ^i^zt  when  a  sample 
of  grade -eligible  students  were  presented  both  items  in  the  given  year,  is 


Xi'X2  -  WT0T(SiS2ri2  +  XiX2) 

where       and  X^  are  the  weighted  sample  standard  deviation  and  mean  based  on 
the  full  sample  of  grade-eligible  students  responding  to  item  1,  S2  and  are 
the  analogous  statistics  based  on  the  full  sample  of  grade- eligible  students 
respondiitg  to  item  2,  and        is  the  appropriately  weighted  sample  correlation 
coefficient  based  on  the  students  responding  to  both  items. 

The  estimation  of  cross-products  between  conditioning  variables  and 
items,  Y'X^,  is  accomplished  in  an  analogous  manner. 

Table  11-6  shows  the  items  used  in  the  writing  trend  scale  (by  supplying 
the  last  three  digits  of  the  ID  number  from  Table  11-1)  and  indicates  the 
elements  of  the  cross-product  matrix  V'V  that  can  be  directly  estimated  at 
each  grade  ai  I  year.    The  entries  in  the  table  are  codes  giving  the  grade  and 
year  for  which  the  corresponding  element  of  item  by  item  cross-product  matrix 
is  estimable.    The  presence  of  a  4,  8,  or  E  indicates  that  the  element  can  be 
estimated  from  the  1984  sample  of  students  ii.  grade  4,  8  or  11,  respectively. 
The  presence  of  an  asterisk  indicates  that  thv,  element  can  also  be  estimated 
from  the  1988  data.     For  example,  the  first  element  in  the  table,  in  the  row 
labeled  003  and  the  column  labeled  003,  indicates  the  grades  and  years  for 
which  the  sum-of -squares  of  nhe  item  N003  can  be  directly  estimated  from  the 
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Table  11-6 

Estimable  Between- item  Correlations 
with  Grade  and  Year  Where  Estimable 


1984  ARM  Items 


Grade  4 
Trend  Items 


Item 

003 
004 
005 
006 
009 
010 

147 
148 

180 
190 
210 


003  004 

8E*  8E* 
48E}? 


Grad*"  11 
Trend  Items 


005 

006 

009 

010 

147 

148 

180 

190  210 

8 

8 

8 

8E 

E 

E  E 

8 

48 

48 

48E 

4 

4 

E 

E  E 

8* 

8* 

8* 

8* 

48* 

48* 

48* 

4 

4* 

48* 

48* 

4 

4* 

48  E* 

4 

4* 

E* 

E*  E* 

4* 

4 

4* 

E* 


E* 
E* 


E* 
E* 
E* 


4:  correlation  estimable  at  grade  4  in  1984 
8:  correlation  esti.^^ble  at  grade  8  in  1984 
E:    correlation  estiraatla  at  grade  11  in  1984 

*  indicates  correlations  estimable  from  198R  data 

y/  indicates  correlations  estimable  from  1988  data  for  grades  8  and  11 
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data.     The  indicates  that  X^'X^  for  this  item  can  be  estimated  for  grade 

8  and  grade  11  students  in  1984,  the  asterisk  indicates  the  X^'X,^  can  also  be 
estimated  for  grade  8  and  grade  11  students  in  1988.    The  "8"  in  the  tiird 
column  of  the  first  row  (under  the  005)  indicates  that  the  cross-product  term 
Xi'Xj  between  items  N003  and  N005  can  only  be  estimated  for  grade  8  students 
in  1984.    The  table  also  indicates  for  which  grades  and  years  the  cross- 
products  between  conditioning  variables  and  item  responses  can  be  estimated, 
any  time  the  item  sum- of -squares  X^'X^  can  be  estimated,  the  conditioning 
variable  by  item  cross-product  vector  Y'X^  can  also  be  estimated. 

Besides  indicating  the  terms  of  the  matrix  V'V  that  are  directly 
estimable  for  a  given  grade  and  year,  Table  11-6  also  indicates  the  terms  that 
are  not:     the  Type  2  and  Type  3  terms.     For  example,  the  sura- of -squares  of  the 
item  N003  is  a  Type  2  term  for  grade  4  students  in  1984  and  in  1988  since  it 
cannot  be  directly  estimated  from  data  from  grade  4  students  but  can  be 
estimated  from  data  from  other  grades.    An  example  of  a  Type  3  element  is  the 
cross-proauct  term  between  items  N003  and  N0147.    The  blank  at  that  position 
in  Table  11-6  indicates  that  this  term  cannot  be  directly  estimated  for  any 
grade  or  year.    This  is  because  items  N003  and  N0147  have  never  been  presented 
together  to  any  student.     The  procedures  for  estimating  Type  2  terms  are  given 
in  the  next  subsection  (11.1.2.4).     Subsection  11.1.2.5  gives  the  procedure 
for  estimating  Type  3  terms. 


11.1.2.4       Estimation  of  Type  2  Elements  of  C 

As  described  above  ,  a  Type  2  element  of  the  cross-product  matrix  for  a 
given  grade  and  year  corresponds  to  a  term  that  is  not  directly  estimable 
based  on  data  for  that  grade  and  year  (the  target  grade  and  year) ,  but  that 
can  be  estimated  based  on  data  from  another  grade  or  another  year  (the  donor 
grade  and  year).     The  estimation  procedure  is  as  -ollows. 

Let  Xi  be  the  set  of  items  held  in  common  between  the  target  grade  and 
year  and  the  donor  grade  and  year  and  let  X2  be  the  set  of  items  presented  to 
the  donor  but  not  to  the  target.     For  notational  convenience,  we  will  operate 
as  if  the  entire  donor  ^nd  target  population  had  been  measured  and  that 
complete  information  by  student  is  available  for  all  items  presented  to  the 
students  grade  and  year.    There  is  no  loss  of  generality  because  only 
estimates  of  the  terms  of  the  cross-product  matrix  are  required. 

The  known  information  for  the  target  population  is  the  matrix 

Vt  -  [Yt  XiJ 

consisting  of  the  conditioning  variables  and  the  items  held  in  common  with  the 
donor  population.    The  known  information  for  the  donor  populition  is  the 
corresponding  matrix 

Vd  -  [Yd 
plus  the  set  of  items  A2d. 
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We  seek  estimates  of  the  Type  2  terms  V^^zt         ^2t  ^2t       ^he  cross- 
product  matrix 


Beaton  and  Johnson  (1987)  show  that  the  estimator 

is  unbiased  under  the  assumption  that  [X^^  X2d)  and  [X^^  X2t]  cire  each 
distributed  with  cominon  variance  matrix  S  and  mean  matrices  Y^B  and  Y^B. 
Under  the  same  assumptions  they  show  that  X2t'X2t  can  be  unbiasedly  estimated 
by 

X2t'X2t  -  X2d'Vd  (v^'v^y  v[y^(v^'v^y'  v^x^^  +  (n  -  g)  j:^., 

where 

G  -      trace  [(V^'V^)"^  V^'VJ  , 

N         is  the  number  of  students  taking  any  pair  of  items 
Sz-i     -  (X2d'X2d  -  X2d'Vd(Vd'Vd)"^  Vd'X2d)/(N^m) 


and 


m 


is  the  number  of  columnn  of  Vj. 


All  estimates  of  Tjrpe  2  elements  were  accomplished  in  the  above  manner. 
Frequently,  there  were  a  number  of  candidates  for  the  donor  population.  The 
following  hierarchy  was  followed  in  selecting  the  population  to  be  the  donor 
for  the  given  target  peculation: 

1)  same  grade  as  the  target  and  the  other  year 

2)  same  year  as  the  target  aad  the  closest  grade 

3)  any  grade  or  year  where  data  was  available 

11.1.2.5       Estimation  of  Type  3  Elements  of  C 

The  final  step  in  the  construction  of  an  estimate  C  of  the  cross-product 
matrix  V'V  for  a  given  grade  and  year  is  the  estimation  of  the  Type  3 
elements.    These  are  cross-product  terms,  X^'Xz,  between  pairs  of  items  that 
were  never  jointly  presented  to  any  student  in  any  grade  or  year.    Such  item 
pairs  are  identified  by  a  blank  in  Tablf  il-6. 


For  a  concrete  example,  consider  the  cross-product  term  between  the 
items  N0003  and  NO 147.    The  item  N0003  was  presented  to  grade  8  and  grade  11 
students  in  1984  and  1988  while  the  item  N0147  was  presented  to  grade  4 
students  in  1984  and  1988.    Consequently,  the  cross-product  between  this  pair 
of  items  is  not  directly  estimable  at  every  grade  and  year  since  the  items 
have  never  been  jointly  presented  to  a  sample  of  students. 

Let  Xi  and  X2  represent  such  a  pair  of  items  and  consider  estimating 
Xi'X2  for  the  target  grade  and  year,  t.    At  most  one  of  the  two  items  can  have 
been  administered  to  the  target  population,  although  it  is  possible  that 
neither  item  was  adirlnistered  to  the    arget  population.    Let  a  index  the  donor 
grade  and  age  for  item  X^  and  let  b  index  the  donor  grade  and  age  for  item  X2. 
(Either  a  or  b,  but  not  both,  might  correspond  to  the  target  population  t.) 

The  known  data  from  the  donor  population  a  are  the  conditioning 
variables,  Y^^,  and  the  scorer  X^^^  on  the  item  1.    The  known  data  from  the 
donor  population  b  are  the  conditioning  variables,  Yj,,  and  the  scores  X2b  on 
the  item  2.    A£;sume  that,  to  a  reasonable  degree  of  approximation, 


and     X2b     ^h^z  ^zh 

where  fii  and  P2         vectors  of  unknown  constants  and  E^^  and  E2b  are  vectors  of 
independent  and  identically  distributed  errors  where  each  element  of  E^^  has 
m^,an  0  and  variance  a^^  and  each  element  of  E2b  has  mean  0  and  variance  a^^- 
Further  assume,  that,  had  the  items  been  presented  to  the  target  population, 
then  the  scores  X^^  and  X2t  on  the  items  for  the  target  population  could  be 
modeled  as 


^21      ^tfii  +  ^21 

where  fii  and  P2  above,  where  the  means  of  E^^^  and  E2t  are  zero,  and 

where  the  variances  of  each  element  of  ^2t  ^^®»  respectively,  a^^  and 

Then,  if  X^t  and  X2t  had  been  jointly  presented  to  the  target  popul::.tion, 
the  expectation  of  Xn'^zt  would  bt, 

Pi^t^tP2  +  N  ^12  (11-5) 
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where        is  the  covariance  between  the  two  items  after  adjusting  for  the 
conditioning  variables.    An  unbiased  estimate  of  the  first  term  of  (11.5)  is 


A  A 


(11.6) 


where  -  (Y.'YJ'^  Y.'X^ 
and      ^2  -  (Yb'Yb)-^  ^^'^7h^ 


An  estimator  of  a^^  i^ 


where  -  [Xi.'Xi.  -  X^.'Y^  (Y.'Y,)-^  Y,'XiJ/(N-p) 
and  -  [X2b'X2b  -  X^b'Yb  (Y^'Y^)"^  Yb'X2b]/(N-p) 

and  where  p  is  the  rank  of  Y^  and  Yj,.    It  remains  to  get  an  estimate  of  r^j, 
the  correlation  between  the  two  Items.    The  estimate  of  r^j  was  based  on  the 
observed  distributions  of  tht.  between  item  correlations  for  the  target  grade 
and  age. 

Let       be  the  vector  of  t^tvreen  item  correlations  for  all  pairs  of  items 
jointly  presented  to  the  target  population  t.    A  typical  element  of  R^^,  r^j, 
is  the  observed  correlation  between  item  i  and  icem  j,  after  adjusting  for  the 
conditioning  variables  Y^.  Let 

-  arctanh(rij)  -  4  log^  {(1  +  -  r^j)) 

be  the  result  of  applying  Fisher's  variance  stabilizing  transformation  to  the 
correlation  r^j  and  let  be  the  vector  of  the  Zj^'s.  The  distribution  of  z^^ 
is  modeled  as 


ZjLj  -  a  +  bi  +  bj  + 


(11  8) 


where 


a 


is  L.  constant  common  to  all  elements  of 


b 


is  a  constant  common  to  all  elements  of       involving  the  item  i; 
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is  a  constant  '^ommon  to  all  elements  of  involving  the  item  j ; 
and 


Cj^j       is  a  normal  random  variable  with  mean  0  and  variance  a|. 

Let  a,  b^,  and  bj  be  the  least- squares  estimates  of  a,  b^  and  bj  in 
(11.8)  and  let  e^j  be  the  least-squares  residual.    The  set  of  all  least- 
squares  residuals  from  the  additive  fit  to       was  assumed  to  be  a  set  of 
normally  distributed  random  variables  with  common  mean  0  and  common  variance 

Let       -  {^ij-  ^>  j  I  items  i  and  j  jointly  presented  to  population  t}  be 
the  set  of  unique  residuals.     (That  is,  the  elements  of  Z^  come  in  pairs, 
there  being  a  value  z^j  and  a  value  Zj^,  which  are  equal,  for  any  pair  of  items 
presented  to  the  target  population.    Consequently,  the  lea.st- square  residuals 
e^j  and  Cj^  are  identical;  only  one  of  these  is  included  in  E^.) 

Let  Ej.  have  K  elements  and  let 


be  the  ordered  values  of  E^.    Under  the  assumptions,  the  e^j^j  are  order 
statistics  of  a  sample  of  size  K  from  a  N(0,  a^")  distribution.    Form  the 
standard  normal  working  values 


Wj  <  W2  <  . . .  <  Wk 

where  w^  is  the  standard  normal  deviate  for  p    "   bility  (i  -  l/3)/(K  +  1/3)  . 
(Wj^  is  a  close  approximation  to  the  median  oT  uie  distribution  of  the  i^^ 
order  statistic  from  a  sample  of  K  standard  i.ormal  random  variables.) 

A  regression  of  the  e^^j's  on  the  w^^'s  produces  estimates  of  the  mean  and 
standard  deviation  of  the  normal  distribution  best  fitting  the  distribution  of 
the  residuals  in  E^:  The  intercept  estimates  the  mean  and  the  slope  estimates 
the  standard  deviation,  The  above  process  was  conducted  separately  for 

each  of  the  s'x  grade-by-year  populations.    It  was  found  in  each  case  that  the 
empirical  distribution  of  the  residuals  was  quite  closely  approximated  by  a 
normal  distribution  with  mean  0  and  standard  deviation       and  that  the  values 
of  the  standard  deviations  for  each  of  the  six  populations  were  nearly  the 
same. 

Accordingly,  the  elements  of  E^^  for  all  six  grade-by-year  populations 
were  pooled  to  form  a  combined  set  of  least- squares  residuals  with  M  elements 
(where  H  is  the  stun  of  the  nt'\er  of  elements  in  each  of  the  E^)  ,  sorted  to 
form  the  order  statistics  e^^^j  <  e^2)  ^  . .  •  ^  e^j^j  and  regressed  against  the 
standard  normal  working  values  for  a  sample  of  size  M.    The  result  of  this 
regression  was  an  intercept  of  0  and  a  slope  of  .09.    The  fit  of  this 
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regression  was  excellent,  with  an      -  .996,  implying  that  the  prediction  of 
the  lessf  j^quares  residuals  by  the  regression  on  the  normal  working  values 
accounted  for  99.6  percent  of  the  variance  of  the  actual  values. 
Consequently,  the  distribution  of  the  least- squares  residuals  from  the  model 
(11.8)  is  closely  approximated  by  a  normal  distribution  with  a  mean  of  0  and  a 
standard  deviation  of  .09,  taken  to  estimate  (Jj  for  all  six  grade -by-year 
populations. 

The  imputation  of  a  value  for  population  t  of        ^^^»  hence,  of  X^'^z* 
for  a  pair  of  items  that  never  appeared  together  was  accomplished  via  the 
following  steps: 

1)  draw  a  random  variable  e^2  ^^^^  ^  normal  distribution  with  mean  0 
and  standard  deviation      ^  .09. 

2)  form  the  imputed  value        -       +  ^itx  +  ^2b  +  ^12 
where 

a^        is  the  least- squares  estimate  of  the  constant  common  to  all 
Fisher-transformed  correlations  in  the  target  population; 

is  the  least-squares  estimate  of  the  constant  common  to  all 
Fisher- transformed  correlations  involving  the  item  1  in  the 
donor  population  a  for  the  item;  and 

is  the  least-squares  estimate  of  the  constant  common  to  all 
Fisher-transformed  correlations  involving  the  item  2  in  the 
donor  populations  b  for  the  item. 

3)  back -trans  form        to  form  the  estimate  of  the  correlation  r^z- 

fi2  -  [exp(2  Z12)  -  l]/[exp(2  Z12)  +  H 

4)  form  the  imputed  value  of  ^i'^2  target  population  by 

Xi'X2  -    ^i'Yt'Yt^2  +  ^^1^2^12 
where  ^1  and  ^2         defined  by  (11.6)  and       and  ^2  are  defined  by 
(11.7). 


11.1.2.6       Application  to  the  Writing  Trend  Scale 

For  each  of  the  thrse  grades  and  each  of  the  two  years,  separate 
estimates  Df  the  matrix  C  were  made.    The  order  of  estimation  was  as  follows. 
First,  all  Type  1  elements  in  all  six  cross-product  matrices  were  estimated. 
The  second  step  was  the  estimation  of  the  Tjrpe  2  elements  for  each  grade  and 
year  based  on  the  Type  1  estimates  from  the  donor  grade  and  year.    The  order 
of  preference  for  selecting  the  donor  population  was:     (1)  same  grade,  (2) 
another  adjacent  grade  in  the  same  year,  and  (3)  any  available  grade  and  year 
Finally,  the  Type  3  elements  were  estimated.    To  allow  for  the  measurement  of 
the  error  due  to  imputation  of  the  between- item  correlations,  five  cross- 
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product  matrices  were  generated  for  each  grade  and  year,  each  matrix  C  being 
based  on  a  different  set  of  draws  from  the  distribution  of  correlations. 

The  resultant  niatrices  were  then  used  as  the  basis  for  constructing  the 
Cq  matrices  as  was  detailed  in  11.1.2.1.    Five  Ce  matrices  were  created  for 
each  grade  and  year,  each  based  on  one  of  the  five  C  matrices.  To 
approximately  account  for  the  effects  of  the  sample  design  and  the  amount  of 
information  available,  each  of  the  Cg  matrices  vjas  scaled  to  be  consistent 
with  a  sample  of  1,000.    For  each  of  the  five  Cq's  for  a  given  grade  and  age, 
a  single  plausible  value  was  computed  for  each  student  according  to  the 
formulas  in  11.1.2.1.    This  resulted  in  five  plausible  values  for  each 
student. 


11.1.3  Other  Ai*alyses  of  Trends  in  Writing  Performance 

In  addition  to  trends  in  primary  trait  scores,  trends  were  also  measured 
for  mechanics  of  writing  and  for  overall  writing  fluency.    Trends  in 
components  of  mechanics  of  writing  at  each  age  were  based  on  a  selected 
writing  prompt  given  to  the  age  group  in  both  1984  and  1988.     The  \vTiting 
items  used  for  the  assessment  of  the  mechanics  of  writing  were  "Spaceship" 
(N014840)  for  grade  4  and  "Recreation  Opportunity"  (N000310)  for  grades  8  and 
11.    All  analyses  were  based  on  representative  subsamples  of  around  500 
responses  to  each  item  at  each  grade  and  year.     In  the  sample  selection,  Black 
students  were  sampled  at  a  higher  rate  to  provide  sufficient  sample  size  to 
allow  for  comparisons  in  performance  between  Black  and  White  students.  The 
student  weights  were  adjusted  to  reflect  this  oversampling  of  Black  students 
by  the  following  poststratif ication  process:     1)  for  each  grade,  the  students 
selected  for  the  writing  mechanics  analysis  were  categorized  by  gender  and  by 
race/ethnicity  (White,  Black,  Hispanic,  other),  producing  eight  cells;  2)  the 
sampling  weights  of  the  students  within  each  cell  were  then  multiplied  by  a 
poststratif ication  factor  computed  as  a  ratio  whose  denominator  is  the  sum  of 
weights  of  all  students  in  the  cell  selected  for  the  mechanics  analysis  and 
whose  numerator  is  the  sum  of  .he  weights  of  all  students  in  the  writing 
assessment  of  the  specified  grade,  gender,  and  race/ethnicity.    All  papers 
used  ir  this  analysis  were  scored  in  1988;  the  actual  sample  sizes  are  shown 
in  T:*ole  11-7. 

Table  11-7 
Sample  Sizes  for  Mechanics  Scoring 

1984*  1988 

Grade  4         Spaceship  506  484 

Grade  8         Recreation  Opportunity  474  517 

Grade  11       Recreation  Opportunity  522  497 


*  All  selected  198A  papers  were  also  rescored  for  primary  trait. 
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Two  writing  items  for  each  grade  in  the  bridge  samples  were  holistically 
scored  for  overall  writing  fluency.    To  allov^  the  measurement  of  trends  in 
overall  writing  fluency,  a  sample  of  responses  in  the  1984  assessment  to  the 
same  items  were  also  holistically  scored.    THie  items  are: 

Grade    4    "Spaceship"  and  "Flashlight" 

Grade  8  "Recreation  Opportunity"  and  "Food  on  the  Frontier" 
Grade  11    "Recreation  Opportunity"  and  "Food  on  the  Frontier" 

Table  11-8  shows  the  sample  sizes  £o^  f.he  measurement  of  trends  in  the  fluency 
of  writing. 


Table  11-8 
Sample  Sizes  for  Holii-tic  Scoring 


1984* 

1988 

Grade 

4 

FlasMight 
Spaceship 

940 
1161 

615 
1257 

Grade 

8 

Food  on  Frontier 
Recreation  Opportunity 

1184 
1286 

1247 
1302 

Grade 

11 

Food  on  Frontier 
Recreation  Opportunity 

1180 
1?.54 

1192 
1182 

*  ALL  L98A  rescored  papers  were  aLso  hoListicalLy  scored. 


11.2    ANALYSIS  OF  THE  CROSS -SECTIONAL  WRITING  DATA 

The  analysis  of  the  1988  writing  cross-sectional  data  was  based  on  the 
data  collected  from  the  respondents  to  the  writing  items  in  the  1988  main 
assessment  and  consists  of  three  components:  (1)  analyses  based  on  the 
responses  to  the  writing  items  in  the  focused-BIB  booklets,  (2)  analyses  to 
examine  the  relationship  bet  ,een  writing  performance  and  writing  process  and 
instruction,  and  (3)  analyses  to  determine  the  effect  of  allocated  time  on 
measured  writing  ability.    All  analyses  were  made  onl^  for  grade-eligible 
students . 


11. 2.1  Primary  Trait  Analysis 

The  major  analyses  of  the  cross-sectional  data  were  based  on  the  primary 
trait  scores  of  the  responses  of  students  to  15  writing  items  presented  in  the 
focused-BIB  portion  of  the  assessment.    The  specific  items  and  the  blocks 
containing  them  are  shown  in  Table  11-9.     For  each  grade  there  are  seven 
blocks  of  writing  items,  where  each  block  typically  contains  a  single  writing 
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Table  11-9 


Assigrjnent  of  Writing  Items  for  Cross -sectional  Writing  Assessment 


Grade 
4 


BIB  Block 
Grade 
8 


Grade 
12 


N0003  Recreation  0pp. 
N0004  Food  On  Frontier 
N0005  Dissecting  Frogs 
N0009  Radio  Station 
N0077  Ghost  Story 
N0147  Plants 
N0148  Spaceship 
Nv^lSO  Space  Program 
N0210  Bike  Lane 
WOOOl  Favorite  Story 
WOO02  Animals 
W0003  Three  Wishes 
W0004  Why  Favorite  Story 
W0005  TV  Habits 
W0006  Personal  Incident 


5 

2 
4 


6 
7 
8 


5 
2 
4 
4 

3 


5 
2 


6 
7 
8 


4 
4 


6 
7 
8 


Table  11-10 


Sample  Sizes  for  Primary  Trait  Analyses 
of  Cross- sectional  Writing  Performance 


Grade 
4 


Grade 
8 


Grade 
12 


K0003  Recreation  0pp. 

N0004  Food  On  Frontier 

N0005  Dissecting  Frogs 

N0009  Radio  Station 

NO077  Ghost  Story 

N0147  Plants 

N0148  Spaceship 

NO 180  Space  Program 

N0210  Bike  Lane 

WOOOl  Favorite  Story 

W0002  Animals 

WO 003  Thres  Wishes 

W0004  Why  Favorite  Story 

W0005  TV  Habits 

W0006  Personal  Incident 


2672 
2015 
2012 
2005 


2687 
2002 
2644 


1950 
1954 
2612 
2612 
1955 


2608 
1954 
2596 


1830 
1800 


1822 


2415 
2415 


2441 
1788 
2407 


TOTAL 


6679 


6525 


6069 
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task.    In  the  focused-BIB  design,  each  student  responds  to  three  of  the  seven 
blocks  of  writing  tasks.    The  number  of  students  responding  to  each  writing 
task  is  shown  in  Table  11-10. 

Books  of  tables  were  created  presenting  population  estimates,  by  item 
of  the  proportions  of  students  achieving  each  of  the  possible  primary  trait 
scores  (0-not  rateable,  1-unsatisfactory ,  2-ininimal,  3-adequate,  4-elaborated) 
as  well  as  the  average  primary  trait  score.     These  results  were  presented  by 
each  of  the  common  background  reporting  variables  as  well  as  by  variables 
relating  to  writing  process  and  instruction. 


11.2.2  The  ARM  Cross-sectional  Writing  Scale 

The  primary  analysis  of  the  cross- sectional  data  was  made  in  terms  of 
average  response  method  writing  scale  scores  based  on  the  15  items  shown  in 
Table  11-9.    The  same  technology  as  wa.-^  used  to  create  the  ARM  trend  writing 
scale  was  used  to  create  this  ARM  cross- sectional  writing  scale,  although  the 
target  sets  of  items  for  the  two  scales  are  largely  different  (there  are  five 
items  in  common  between  the  15  items  making  up  the  cross -sectional  scale  and 
the  11  items  making  up  the  trend  scale). 

The  steps  used  to  generate  ARM  plausible  values  for  the  cross- sectional 
writing  sct^le  are  the  same  as  those  for  che  writing  trend  scale  detailed  in 
subsection  11.1.2.    Analogous  to  the  waiting  trend  scale,  the  basis  for  the 
estimation  of  a  plausible  value  for  the  cross -sectional  scale  is  an  estimate  C 
of  the  full  sums-of -squares-and-cross-products  matrix 


V'V  - 


Y'Y  Y'X 
X'Y  X'X 


where,  as  before,  Y'Y  is  the  sums-of -squares-and-cross-products  matrix  of  the 
conditioning  vt^riables,  X'X  is  the  suras-of-squares-and-cross-products  matrix 
of  the  15  items  in  the  cross-sectional  assessment,  and  X'Y  is  the  cross- 
product  matrix  betveen  the  items  and  the  conditioning  variables.     For  the 
construction  of  tho  cross-sectional  scale,  three  separate  estimates  C  of  the 
suias-of -squares-and-cross-products  matrix  were  created:  one  for  each  of  the 
grades  4,  8,  and  12. 

The  conditioning  matrix  Y  for  the  cross-sec',  onal  assessment  contains 
the  values  of  53  common  core  conditioning  variables  (listed  in  Table  C-1  in 
Appendix  C)  measuring  each  student's  status  on  a  variety  of  demographic, 
background,  and  attitude  questions.    Tue  matrix  Y  also  contains  30  variables 
related  to  each  student's  attitudes  toward  and  experiences  in  writing.    A  list 
of  these  conditioning  variables  appears  as  Table  C-4  in  Appendix  C. 

As  with  the  trend  scale,  the  elements  of  the  estimate  C  of  V'V  for  a 
given  grade  fall  into  three  general  types:  Type  1  elements  directly  estimable 
from  available  data  for  that  grade;  T^^pe  2  elements  estimable  from 
relationships  observed  in  another  grade;  and  Type  3  elements  involving  items 
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never  jointly  presented  to  any  student  and  requiring  the  imputation  of  between 
item  correlations. 


The  Type  1  elements  include  the  sums-of -squares  matrix  Y'Y  of  the 
conditioning  variables,  the  elements  of  the  matrix  X'Y  corresponding  to  the 
cross-products  of  items  with  conditioning  variables  for  the  items  presented  to 
that  grade,  and  the  elements  of  the  matrix  X'X  corresponding  the  sums-of- 
squares  of  items  and  cross-products  between  items  presented  to  that  grade. 
The  estimation  procedure  for  the  Type  1  elements  of  C  was  identical  to  that 
given  in  subsection  11.1.2,3. 

Table  11-11  shows  the  items  used  in  the  cross- sectional  writing  scale 
and  indicates  the  elements  of  the  matrix  V'V  that  can  be  directly  estimated  at 
each  year.    Analogous  to  Table  11-6,  the  entries  in  Table  11-11  are  codes 
giving  the  grades  for  which  the  corresponding  element  of  the  item-by- item 
cross-product  matrix  is  estimable.    The  presence  of  a  4,  8,  or  T  indicates 
that  the  element  can  be  directly  estimated  from  the  1988  cross  sectional 
sample  of  students  in  grade  4,  8,  or  12,  respectively.     The  table  also 
indicates  for  which  grades  the  cross-products  between  conditioning  variables 
and  item  responses  can  be  directly  estimated:    Any  time  an  item  sum- of -squares 
can  be  directly  estimated,  wO  can  the  cross-products  between  that  item  and  the 
conditioning  variables. 

In  addition  to  indicating  the  directly  estimable  terms  (the  Type  1 
terms),  the  table  also  indicates  the  Type  2  and  the  Type  3  terms.     The  Type  2 
terms  for  a  given  grade  are  identified  by  the  absence  of  the  code  for  that 
grade  but  the  presence  of  a  code  for  a  different  grade.     The  Type  3  terms  are 
identified  by  the  absence  of  codes  for  all  three  grades.     The  estimation 
procedure  for  the  Type  2  terms  was  identical  to  that  given  in  11.1.2.4  and  the 
procedure  for  estimating  the  Type  3  terms  was  identical  to  that  given  in 


Separate  estimates  of  the  matrix  C  were  made  for  each  grade  using  the 
following  order  of  estimation.    First,  all  Type  1  elements  in  all  three  cross- 
product  matrices  were  estimated  followed  by  the  estimation  of  the  Type  2 
elements  for  each  grade  based  on  the  Type  1  estimates  from  the  donor  grade. 
When  possible,  the  adjacent  grade  was  selected  as  the  donor  grade.  Finally, 
the  Type  3  elements  were  estimated.    Five  cross-product  matrices  were 
generated  for  each  grade  where  each  matrix  C  was  based  on  a  different  set  of 
draws  from  the  distribution  of  correlations.     This  allowed  for  the  measurement 
of  the  error  due  to  imputation  of  the  missing  between- 1*" em  correlations. 

The  resultant  matrices  fortaed  the  basis  of  the  Cq  matrices  described  in 
11.1.2.1.     Five  Cq  matrices  were  created  for  each  grade,  where  each  of  these 
was  based  on  a  different  on  of  the  five  C  matrices  for  that  grade.  To 
approximately  account  for  the  sample  design,  each  Cq  matrix  was  scaled  to  be 
consistent  with  a  sample  of  1,000.    For  each  of  the  five  C^'s  for  a  given 
grade,  a  single  plausible  va^ue  wa.^  generated  for  each  student  via  the 
equations  in  11.1.2.1  resulting  in  five  plausible  Vc^ilues  for  each  student. 


11.1.2.5. 


Table  11-11 

Estimable  Between- lte::i  Correlations 
for  the  Cross -sectional  Scale 
with  Grade  Whore  Estimable 


Item 

N0003 

N0004 

N0005  N0009 

N0077  N0147  N0148 

N0180 

N0210 

WOOOl 

W0002 

W0003 

HO  004 

W0005 

W00( 

N0003 

8T 

8T 

8  8 

8T 

T 

T 

8T 

8T 

8T 

NO  004 

8T 

8  8 

8T 

T 

T 

8T 

8T 

8T 

NO  005 

8  8 

8 

8 

8 

8 

NO  009 

48 

48       4  4 

4 

4 

4 

8 

8 

8 

N0077 

48T      4  4 

T 

T 

4 

4 

4 

8T 

8T 

8T 

NO  147 

4  4 

4 

4 

4 

NO  148 

4 

4 

4 

4 

N0180 

T 

T 

T 

T 

T 

NO  210 

T 

T 

T 

T 

WOOOl 

4 

4 

4 

WO  002 

4 

4 

W0003 

4 

W0004 

8T 

8T 

8T 

W0005 

8T 

8T 

WO  006 

8T 

4:  correlation  estimable  at  Grade  4 
8:  correlation  estimable  at  Grade  8 
T:  correlation  estimable  at  Grade  12 


293  309 

o  308 


11.2.3  The  Effect  of  Block  Position  on  Writing  Performance 

In  the  course  of  the  analysis  of  the  cross-sectional  writing  data,  it 
was  discovered  that  the  performance  on  each  writing  task  appeared  to  be 
related  to  the  position  of  that  task  in  the  assessment  booklet.     In  the 
focused-BIB  design  used  for  the  1988  writing  cross-sectional  assessment,  each 
block  cf  writing  tasks  appeared  in  three  distinct  booklets:    once  as  the  first 
cognitive  block,  once  as  the  second,  and  once  as  the  third.     Since  the  samples 
of  students  responding  to  each  booklet  are  themselves  representative  Siimples 
of  the  population,  it  would  be  expected,  all  other  things  being  equal,  that 
the  scores  of      /.dents  who  were  presented  the  task  as  the  first  block  in  their 
assessment  booklet  would  be  closely  comparable  to  the  scores  uf  students 
presented  in  the  block  second  or  third. 

Table  11-12,  which  gives  the  mean  primary  trait  score  across  items  by 
grade  and  block  position,  shows  that  this  is  not  the  case.    While  the  mean 
primary  trait  score  for  items  when  they  appear  in  the  first  block  of  a  booklet 
are  not  significantly  different  from  the  means  of  the  items  when  they  appear 
in  the  second  block,  both  means  are  significantly  higher  than  the  means  of  the 
items  when  they  apnear  in  the  last  block.    Tlie  effect  occurs  at  every  grade 
although  it  appears  to  be  strongest  for  grade  12.     The  same  phenomenon  occurs 
for  many  of  the  individual  items. 

Table  11-12 

Mean  Primary  Trait  Score  by  Position  of  Item  in  '^ooklet* 

1st  block  2nd  block  3rd  block 

Grade  4         2.00(.02)     '  2.04(.02)  1.93(.03) 

Grade  8         2.08(.02)  2.05(.02)  1.99(.03) 

Grade  12       2.20(.02)  2.20(.03)  2.C6(.04) 


*  standard  errors  in  parentheses 

These  results  imply  that  the  estimates  of  writing  performance  based  on 
averaging  across  the  three  positions  may  provide  underestimates  of  the  true 
writing  performance  of  the  students.    The  lower  performance  on  the  third  block 
of  exercises  may,  perhaps,  be  due  to  fatigue.     Further  investigation  is 
planned  of  this  phenomenon,  which  also  occurs  in  the  1988  cross -sectional 
reading  assessment  but  not  in  the  1988  cross -sectional  civics  assessment,  is 
planned. 


11.2,4  Relating  Writing  Performance  to  Writing  Process  and  Instruction 

Analys^es  were  conducted  to  determine  the  association  between  the  ARM 
scale  scores  and  the  composite  indices  relating  to  writing  process  and 
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instruction  derived  from  the  responses  of  the  students  to  the  writing 
background  questionnaire.    The  indices  examined,  and  the  grades  at  which  they 
were  available,  are  as  follows: 


Index 
Enjoy  writing 

Instruction  on  Writing  Process 

Revision 

Planning 


Grades 

4,  8,  12 
8,  12 
12 
12 


Tlie  questions  included  in  each  of  the  composites  are  given  in  Appendix 
D,  along  with  the  values  assigned  to  each  of  the  original  responses  to  derive 
the  cocposites. 

Writing  achievement  was  related  tc  responses  obtained  to  a  teacher 
questionnaire  completed  by  the  writing  teachers  of  roughly  85  percent  of  the 
assessed  eighth-grade  students.    Among  other  information  available  in  the 
teacher  questionnaire  was  the  instructional  techniques  used  by  the  teacher  for 
the  assessed  student  in  particular.     It  was  thus  possible  to  examine  how 
various  instructional  practices  relate  to  measured  writing  achievement,  and, 
since  certain  of  these  questions  are  paralleled  in  the  student  background 
questionnaire,  to  examine  the  relationship  between  instructional  techniques 
reported  used  by  the  teacher  and  the  student's  perceptions  of  the 
instructional  techniques  used. 


11.2.5 


Effect  of  Time  Allocated  to  the  Wiriting  Task 


The  final  analysis  of  the  1988  cross- sectional  data  was  designed  to 
determine  the  effect  of  time  allocated  to  the  writing  task  on  the  estimate  of 
writing  ability.    This  analysis  was  based  on  the  comparison  of  /riting  perfor- 
manc(;  for  two  equivalent  samples  of  students  in  which  one  sample  receives 
twice  as  much  time  as  the  other  to  complete  the  same  writing  exercise.  The 
data  at  each  grade  consist  of  the  responses  to  three  e..ercires,  one  from  each 
of  the  three  major  purposes  of  writing  (informative,  imaginative,  persuasive). 
To  improve  the  measure  of  the  effect  of  allocated  time  on  writing  performance, 
the  analysis  controlled  for  the  score  of  another  writing  exercise  of  the  same 
type,  also  given  to  both  samples  of  students.     (For  grade  8/age  13  and  grade 
12/age  17,  there  were  two  control  exercises  for  ll.e  persuasive  task.)  For 
each  grade/age,  three  writing  booklets  were  administered,  with  one  booklet  for 
each  of  three  categories  of  writing  purposes.    The  booklets  had  the  following 
form: 


Persuasive: 
Informative : 
Imaginative ; 


L3 


The  first  task  (51,82,83)  in  each  booklet  was  allocated  the  usual  time  for 
completion.    The  second  task  (Li,!^,!^)  was  allocated  twice  the  usual  time. 
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The  particular  writing  exercises  for  the  analysis,  along  with  their  control 
exercises,  are  as  follows: 


Grade  Task 


Coatrol  Exercise 


Analysis  Exercise 


Informative 
Imaginative 
Persuasive 


W000140 
W000340 
N000940 


W000220 
N007720 
N014820 


Informative 
Imaginative 
Persuasive 


W000410 
W000610 

N000950  and  N000550 


W000530 
N007730 
N000330 


12 


Informative 
Imaginative 
Persuasive 


W000410 
W000610 

N021050  and  N018050 


W000530 
N007730 
N000330 
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Chapter  12 

DATA  ANALYSIS  FOR  THE  ClVr'^S  ASSESSMENT^ 


Nancy  Allen 
Educational  Testing  Service 


This  chapter  describes  the  analyses  carried  out  on  the  responses  to  the 
cognitive  and  background  items  in  the  1976  and  1982  assessments  of 
ci  Izenship/social  studies  and  the  1988  assessment  of  civics.  These  analyses 
led  to  the  results  presented  in  The  1988  Civics  Report  Card:    Trends  In 
Achievement  from  1976  to  1988  at  Ages  13  and  17,  and  Achievement  in  1988  at 
Grades  4,  8,  and  12  (Anderson,  Jenkns,  Leming,  MacDonald,  Mullis,  Turner,  & 
Wooster,  1990).    The  emphasis  of  this  chapter  is  on  the  methods  and  results  of 
procedures  used  to  develop  the  IRT-based  scale  scores  thac  formed  the  basis  of 
that  report.    However,  some  attention  is  given  to  the  analysis  of  open-ended 
items  and  mean  percents  correct  for  groups  of  items  as  reported  in  The  1988 
Civics  Report  Card.     The  theoretic  underpinnings  of  the  IRT  and  plausible 
value  methodology  described  in  this  chapter  are  given  in  Chapter  9. 

The  techniques  required  to  develop  scale  scores  for  the  cross -sectional 
analysis  of  the  data  from  the  1988  main  BIB-spir:*l  assessment  were  different 
from  the  techniques  required  to  develop  scale  scores  for  the  analysis  of 
trends  in  civics  achievement.    Accordingly,  these  two  analyses  are  presented 
in  separate  sections.     Section  12.1  pertains  to  the  scaling  of  the  data  from 
the  main  assessment;  section  12.2  contains  information  about  the  scaling  of 
the  data  from  the  trend  bridges. 


12.1    CROSS-SECTxONAL  DATA  ANALYSIS 

The  data  from  the  main  BIB-spiraled  assessment  of  civics  in  1988  were 
used  for  cross-sectional  analyses  comparing  the  levels  of  civics  achievement 
for  various  subgroups  of  the  1988  target  populations.    The  main  assessment 
included  two  parts:     the  focused-BIB  samples  and  the  intercorrelation  samples. 
It  included  three  student  cohorts:     students  who  were  cither  in  the  fourth 
grade  or  9  years  old,  students  who  were  either  in  the  eighth  grade  or  13  years 
old,  and  students  who  were  either  in  the  twelfrh  grade  or  17  years  old.  The 
birch  date  ranges  for  age -eligible  students  were  based  on  the  1978,  1974,  and 
1970  calendaiL  years  respectively  for  ages  9,  13  and  17.    The  sampled  students 
in  each  of  these  three  cohorts  were  assesse  '  either  in  the  winter  or  the 
spring.    The  samples  in  the  main  assessment  are  listed,  along  with  the  bridge 


^DatG  analysis  and  scaling  were  performed  by  John  J.  Ferris,  Edward 
Kulick,  Jennifer  Nelson,  Norma  Norris,  Kate  Pashley,  and  Minhwei  Wang.  Eugene 
Johnson,  Robert  Mis levy,  and  Kentaro  Yamamoto  consulted  on  IRT  scaling  and 
generation  of  plausible  values. 
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(or  trend)  samples,  in  Table  12-1.  (See  Chapter  1  and  Chapte>-  3  for 
descri;iions  of  the  target  populations  and  the  sample  design  used  for  the 
assessment.) 


The  pool  of  items  used  i^  the  1988  civics  assessment  contained  a  range 
of  open-ended  and  multiple-choj.ce  questions  measuring  performance  on  sets  of 
objcrtives  documented  in  Civics  Objectives ,  1988  Assessment  (NAEP,  1987b). 
The  objectives  framework  is  described  in  Chapter  2     A  total  of  220  (one  with 
two  parts)  distinct  civics  items  addressing  these  objectives  wa,<»  administered 
in  1£88  using  the  BIB-spiral  design  to  allocs  ^  the  items  to  the  assessed 
students.    All  220  items  were  administered  to  the  maiii  focused- BIB  sample  and 
a  subgroup  of  items  was  administered  to  the  intercorrelation  sample. 

The  "grade-only"  portion  of  the  main  focused-BIB  civics  samples  (whether 
the  time  ot  assessment  was  winter  or  spring)  provided  the  cross-sectional 
results  that  are  reported  in  The  1988  Civics  Report  Card.     In  these  samples, 
each  student  was  administered  a  booklet  containing  three  blocks  of  civics 
cognitive  items,  a  block  of  background  questions  common  to  all  b:':.klets  for  a 
particular  grado/oge  Isvel,  and  a  block  of  civics -related  background  questions 
common  to  all  civics  booklists  for  a  particular  grade/age  level.     Seven  blocks 
of  civics  cognitive  questions  were  administered  at  grade  8/age  13  and  grade 
12/age  17  in  a  total  of  seven  booklets  for  each  age.    At  grade  4/age  9,  three 
blocks  of  civics  cognitive  questions  vere  administered  in  only  oi  3  order  in 
one  booklet.     The  sample  sizes  and  number  of  items  for  the  focused-BIB 
grade/age  samples  are  listed  in  Table  12-2. 

The  purpose  of  the  intercorrelation  samples  is  to  provide  proficiencies 
for  reading,  civics,  U.S.  history,  and.  at  grade  12/age  17,  geography  for  the 
same  samples  of  students.    These  results  are  available  for  secondary  analyses. 
In  the  intercorrelation  samples,  each  student  vho  recei"   d  civics  items  was 
adrinistered  a  booklet  containing  three  blocks  of  subject  area  cognitive  itemo 
i  ^eluding  one  block  of  civics  items  that  was  also  administered  to  the  main 
focused-BIB  samples,  as  well  as  a  block  of  background  questions  common  to  all 
bookie*  .  for  a  particular  grade/age  level  and  /  block  of  background  questions 
selected  from  those  given  to  students  assessed  in  the  reading,  civics,  U.S. 
history,  and  geography  (at  grade  12/age  17)  main  focused-BIB  samples.  Three 
blocks  of  rWics  cognitive  items  were  administered  in  three  booklets  at  grade 
4/age  9  and  grade  8/age  13.     Because  of  the  additional  subject  area, 
geography,  only  two  blocks  of  civics  cognitive  i^ems  were  administered  in  two 
booklets  at    -ade  12/age  17.    The  sample  sizes  and  number  of  items  foi.  .,he 
intercorrelation  grade/age  samples  are  listed  in  Table  12-2. 

It  should  be  noted  that  one  block  aditlnistered  to  the  main  focused-BIB 
samples  at  grade  8/age  13  and  grade  12/age  17  consisted  of  one  open-ended 
item.    This  two-part  item  entiiiled  identifying  the  President  and  discussing 
presidential  responsibilities.    Each  part  of  the  item         scored  separately. 
Both  p,irts  were  scaled  with  the  multiple-choice  items,  but  they  were  also 
analyzed  independently. 

The  next  sections  contain  in  some  detail  a  description  of  the  analysis 
performed  using  the  main  BIB-spiraled  samples.    As  is  usual  in  NAEP  analyses, 
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Table  12-1 


NAEP  1988  Civics  Samples 


Subject 

Booklet 

Time  of 

Age 

Modal 

Sample 

Samole  Code 

Samole  T/pe 

Areas 

Numbers 

Cohort 

Assessment 

Zf' wl.  11 

Dxze 

9  (Main-Civ] 

BIB  Main 

c 

16 

Grade  4/age  9 

Winter, 

spring 

CY 

4 

2,652 

13  [Main- Civ] 

BIB  Main 

c 

22-28 

Grade  8/age  13 

Winter, 

spring 

CY 

8 

5,981 

17  [Main- Civ] 

BIB  Main 

c 

22-28 

Gradp  19  /ap-p  17 

Winter, 

spring 

rv 

xz 

D  ,  0o3 

9[Main-Int] 

BIB  Intercorrelation 

R.C.H 

17-19 

Grade  4/age  9 

Winter , 

spring 

CY 

4 

2,638 

13[Main-Int] 

BIB  Intercorrelation 

R.c.H 

29-31 

Grade  8/age  13 

Winter , 

spring 

CY 

8 

2,590 

17(Main-Int] 

BIB  Intercorrelation 

C , H, G 

30-32 

Grade  12/age  17 

Winter, 

spring 

CY 

12 

2,438* 

13[BR-Civ] 

Bridge  to  1976,  1982 

C 

90 

Age  13 

Fall 

CY 

8 

1,938 

17  [BR- Civ] 

Bridge  to  1976,  1982 

C 

90 

Age  17 

Spring 

Not 

CY  11 

1,786 

*  Booklet  30  did  not  contain  a  civics  block;  the  number  of  students  that  were  administered  a  civics 
block  in  this  sample  is  1,621. 


R  -  Reading 

C  -  Civics 

H  -  U.S.  history 

G  -  Geography 


CY  -         Calendar  year:    Birth  dates  in  1978,  1974, 
and  1970  for  ages  9,  13,  and  17 

not  CY  -  (Age  17  only) :    Birth  dates  betwe-in 

October  1,  1970  and  September  30,  1971 


315 


299 


IB 


Table  12-2 


Sample  Sizes  and  Number  of  Items 
for  the  Main  Assessment  Grade/Age  Civics  Samples 


No.  of  Multiple-  No.  of  Open- 
Sample                   Sample  Size           Choice  Items  Ended  Items 

Grade  4/Age  9 

Main  Focused-BIB  2,652  51  0 

Intercorrelation  2,638  53  0 


Grade  8/Age  13 

Main  Focused-BIB  5,981  153  (151)*  1 

Intercorrelation  2,590  72 


Grade  12/Age  17 

Main  Focused-BIB  5,683  150  (148)*  1 

Intercorrelation  1,621  49  0 

Total  21,165  219  1* 


*    Numbers  of  items  in  the  final  scale  are  included  in  parentheses 
when  different  from  the  number  in  the  assessment. 

Scored  in  two  parts. 


the  process  began  with  an  examination  of  the  i'lems  and  blocks  of  items.  This 
was  followed  by  an  assessment  of  the  dimensionality  of  the  civics  items  in  the 
main  assessment.    The  estimation  of  item  parameters  for  the  unidimensional 
civics  scale  was  completed  next,  followed  by  the  generation  of  plausible 
values.    Finally,  the  plausible  v  i.ues  were  transformed  to  the  final 
proficiency  scale  and  points  on  the  proficiency  scale  were  anchored.  This 
section  of  the  chapter  closes  with  information  about  derived  background 
variables,  scoring  of  the  open-ended  items,  and  content  grouping  of  the  items. 

12.1.1  Item  Analysis  and  Dimensionality  Assessment 

Table  12-3  shows  the  number  of  items,  KR-20  reliability,  mean  number 
correct,  standard  deviation,  and  mean  percent  correct  for  each  block.  These 
values  were  calculated  for  the  weighted  item  results  of  all  multiple-choice 
items  within  a  block,  whether  they  were  used  in  the  scaling  process  or  not. 
The  table  also  gives  the  number  of  students  who  were  administered  the  block 
and  the  percent  not  reaching  the  last  item  in  the  block.    The  results  for  the 
blocks  administered  to  each  grade/age  level  indicated  that  the  blocks  differ 
in  number  of  items,  average  difficulty,  reliability,  and  percent  not  reaching 
the  last  item,  and  so  are  not  parallel  to  one  another. 

Table  12-4  contains  information  about  the  effect  of  the  position  of 
blocks  within  booklets  on  the  average  percent  correct  for  items  within  each 
block  pres'^nted  to  the  focused-BIB  samples  at  grade  8  and  grade  12.    The  three 
blocks  presented  to  .^rade  V^ge  9  students  in  the  focused-BIB  sample  were 
presented  in  only  one  booklet  (in  one  order),  and  so  are  not  included  in  the 
table.    The  averages  for  the  grade-only  portion  of  the  focused-BIB  samples 
show  that  the  order  of  blocks  within  booklets  did  not  have  a  large  or 
consistent  effect  on  proficiency  in  the  civics  focused-BIB  assessment. 

Content  area  experts  considered  the  civics  items  to  be  fundamentally 
related  to  one  another  and,  therefore,  recommended  a  unidimensional  scale. 
The  purpose  of  a  dimensionality  analysis  was  to  make  sure  that  this  decision 
appeared  reasonable  in  light  of  the  data.     So,  the  full- information  factor 
analysis  method  developed  by  Bock,  Gibbons,  and  Muraki  (1985)  was  applied  to 
the  data  using  the  TESTFACT  program  (Wilson,  Wood,  &  Gibbons,  1983). 

Initially,  blocks  of  items  at  each  grade/age  level  were  analyzed 
separately.    One-,  two-,  three-,  and  four-factor  solutions  were  examined. 
Only  the  results  of  che  two- factor  solutions  are  presented  here,  because  the 
first  two  factors  of  higher-order  solutions  were  similar  to  the  factors  in  the 
two-factor  solutions. 

The  dimensionality  results  for  the  civics  blocks  are  shown  in  Table 
12-5.     Four  blocks  had  items  that  contributed  to   ^eywood  cases,  making 
estimation  of  a  solution  impossible.    Those  items  were  deleted  from  the 
dimensionality  analysis.     In  each  block,   the  amount  of  variance  explained  by 
the  first  factor  in  a  two -factor  solution  ranged  from  18  to  34  percent. 
Although  the  results  show  that  the  data  are  not  strictly  unidimensional,  the 
percent  of  variance  explained  by  the  first  factor  was  much  larger  than  the 
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Table  12-3 


Descriptive  Statistics  for  Main  Assessment  Civics  Blocks 


Mean 

nean 

incercor * 

Percint  Not 

Number 

Number 

Percent 

rocu3eci*iiio 

xexacion 

Reaching  the 

Grade /Age 

or  xuenis 

vo  on 

Correct 

c  n 

Correct 

li 

Last  Item 

uZ 

.  /  J 

10 

4 

0 .  U 

CO 

.  by 

O^  CO 

Zo5Z 

O07 

2 

.1 

C3 

16 

.72 

9 

3 

3.2 

.58 

2652 

884 

5 

.1 

r»/. 

.  /  / 

10 

2 

A  O 

.  51 

CO 

zo5Z 

00  C 
OOJ 

3 

.5 

0  /I  Q 

r'O 

oil 
Zo 

Q1 

.  oX 

14 

0 

J .  U 

^A 
.  JH 

2 

.7 

Lp  J 

OA 
Zh 

Q1 

17 

0 

A  *X 

71 
.  /  X 

Z3/!> 

A 

U 

1 

.9 

19 

4 

^  Q 
J  .  o 

Z5o  J 

A 

U 

2 

.1 

07 

z/ 

Q1 
.  oX 

15 

.7 

ft 

.  3o 

O  RO"7 

ZjZ/ 

A 

u 

1 

.6 

do 

OA 
Zh 

.  /o 

11 

.6 

A  «; 

A  Q 
.  HO 

O  C  CO 

Z55  J 

Q7Q 

6 

.3 

OO 
ZZ 

70 

.  /z 

10 

.3 

A  1 
H  .  X 

A  7 

Q 

% 

1 

C8 

1 

2547 

0 

12/17 

C2 

26 

.86 

18 

.5 

5.: 

.71 

2436 

0 

1 

.4 

C3 

26 

.85 

13 

.6 

5.4 

.52 

2441 

0 

2 

.4 

C4 

25 

.82 

14 

.3 

5.0 

.57 

2434 

815 

5 

.0 

C5 

27 

.85 

19 

.5 

4.9 

.72 

2442 

0 

0 

.9 

C6 

24 

.84 

15 

.2 

5.0 

.63 

2426 

806 

3.8 

C7 

22 

.84 

14.1 

4.9 

.64 

2441 

0 

2.2 

C8 

1 

2429 

0 
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Table  12-4 


Effect  of  Block  Position  on  Average  Item  Percent  Correct 
for  the  Main  Focused-BIB  Civics  Samples 

Position  Maximum  Absolute 

Block*  __1  2  3_  Difference 


Grade  8 


C2 

57 

57 

55 

2 

C3 

75 

74 

74 

1 

C4 

67 

68 

68 

1 

C5 

62 

61 

59 

3 

C6 

53 

51 

50 

3 

C7 

50 

49 

49 

1 

C8 

66 

65 

64 

2 

Grade  12 


C2 

74 

75 

72 

3 

C3 

54 

55 

56 

2 

C4 

61 

60 

61 

1 

C5 

75 

75 

74 

1 

C6 

67 

67 

66 

1 

C7 

69 

67 

67 

2 

C8 

77 

75 

76 

2 

*  Note:    A  full  BIB  configuration  was  not  implemented  at  grade  4/age  9 
blocks  C2,  C5,  C6,  C7,  and  C8  are  identical  for  grades  8  and  12.    Except  in 
these  cases,  identity  of  block  numbers  across  grades  dees  not  imply  identity 
of  blocks.    Standard  errors  for  mean  percents  correct  are  approximately  0.5. 
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Table  12-5 


Results  of  the  Full-Information  Factor  Analyses  of 
Tetrachoric  Matrices  of  Civics  Item  Responses 

Percent  of  Explained  Variance 

Gra<'.i^(/Age       Block            Items       Deleted  Itecs     First  Factor  fJ'scond  Factor 

V9               C2                 15                  0                       32  5 

27  5 

22  9 

8/13              C2                 26                   1                        24  3 

30  4 

23  6 

24  4 
20  2 
18  2 


12/17             C2                26                  0                       34  4 

32  4 

27  3 

31  4 

29  3 

3'  2 


Number  of 

Block 

Itrems 

Deleted 

C2 

15 

0 

C3 

16 

0 

C4 

20 

0 

C2 

26 

1 

C3 

24 

0 

C4 

30 

0 

C5 

27 

0 

C6 

24 

0 

C7 

22 

3 

C8 

1 

C2 

26 

0 

C3 

26 

0 

C4 

25 

1 

C5 

27 

0 

C6 

24 

2 

C7 

22 

0 

C8 

1 
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percent  of  variance  explained  by  the  second  factor,  and  the  factor  analytic 
solutions  had  no  clear  interpretation  in  terms  of  the  item  content  categories. 
This  supported  the  use  of  a  unidimensional  scale  to  sunmiarize  the  results. 


12.1.2  Estimation  of  Item  Parameters 

The  computer  program  BILOG  (Mislevy  &  Bock,  1982)  was  used  to  estimate 
the  item  parameters  of  the  three-parameter  IRT  model  for  217  of  the  219 
mu"' tiple-choice  items  and  the  two  sections  of  the  open-ended  item  (for  a  total 
of  219  items  in  the  final  :;cale),  using  a  random  subsamjl<i  of  7,859  of  the 
21,165  students  in  the  main  assessment  samples.    Most  items  had  approximately 
1,000  responses  in  the  subsample.    The  actual  range  of  responses  per  item  was 
from  972  to  2,656.    After  examination  of  the  items  for  differential  item 
functioning  across  groups,  students  from  all  three  grade/age  groups  and  from 
both  the  main  focused- BIB  and  intercorrelation  samples  were  Included  in  the 
scaling  process.    The  responses  were  not  weighted.  (See  Chapter  9  and  Beaton, 
1987b,   for  further  descriptions  of  the  scaling  process.) 

The  two  items  that  were  dropped  from  the  scale  were  excluded  because  of 
lack  of  fit  to  the  IRT  model.     Neither  of  these  items  had  monotonically 
increasing  empirical  item  characteristic  curves.    The  NAEP  item  numbers  for 
these  items  are  P005804  and  P006001.    Thoy  were  the  13th  and  15th  items  in 
block  C5  for  both  grade  8/age  13  and  gr;'.de  12/age  17.    Other  than  appearing  in 
the  same  block,  the  items  were  an>::ilatf.d.    No  items  were  dropped  because  of 
their  differential  item  f«i»ctior.ing  vith  respect  to  the  different  age  groups, 
the  two  types  of  samples,  the  tw^^  gender  groups,  or  the  several  racial/ethnic 
groups . 

Of  the  217  multiple -choice  items  included  in  the  final  scale,  64  had 
only  two  choices,  or  two  choices  and  an  "I  don't  know"  alternative. 
Consequently,  these  items  had  low  discrimination  parameter  values  with  high 
"guessing**  parameter  values  (about  .50). 

In  the  final  scale,  there  were  36  items  in  common  for  grade  4/age  9  and 
grade  8/age  13,  and  99  items  in  common  for  grade  8/age  13  and  ^rade  12/age  17. 
There  were  no  it  ms  in  common  across  all  three  grade/age  groups.  Fifteen, 
eighteen,  and  fifty-one  items  were  administered  only  at  grade  4/age  9,  grade 
8/age  13,  and  grade  12/age  17,  respectively. 

Taule  F-3  in  Appendix  F  lists  the  estimated  item  naranieters.    Tliese  item 
parameter  estimates  are  direct  output  from  the  BILOG  program  specifying  three 
subpopulations,  prior  to  any  rescaling.     BecAU^>e  of  the  Indeterminacy  of  the 
IRT  scale,   the  origin  and  size  of  the  scale  were  set  pro^'isionally  by 
standardizing  the  distribution  of  the  calibration  sample  of  examinees  to  have 
a  inean  of  zero  and  a  variance  of  one. 


12.1.3  Generation  of  Plauiiible  Vai"'is 

Univariitfc  plausible  values  were  gcnerat<5d  using  the  M-GROUP  compute! 
program  (Sheehan,  1985)  according  to  the  conditioning  procedure  described  in 
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Chapter  9.    The  final  student  weights  were  used  at  this  stage  ot  .he  analysis. 
Conditioning  was  conducted  separately  for  each  of  the  three  grade/age  groups 
and  for  the  mcln  focused- BIB  and  intercorrelation  samples  because  different 
conditioning  background  variables  were  available  for  each  group.    The  weighted 
means  and  standard  deviations  of  the  plausible  values  for  the  main  focused- BIB 
were  compared  to  those  for  the  intercorrelation  samples  (Table  12-6).  Because 
the  means  for  the  two  types  of  samples  were  disparate,  and  because  at  grade 
8/age  13  and  grade  12/age  17  the  focused-BIB  assessment  balanced  the  order  of 
administration  of  the  blocks,  only  results  for  the  "grade-only"  portion  of  the 
focused-BIB  samples  are  reported  in  The  198S  Civics  Report  Card. 


Table  12-6 

Civics  Means  and  Standard  Deviations  ^^n  the  Calibration  Scale 
for  Grade/Age  Mai  x  Assessment     'B  Spiral  Samples 

First  All  Five 

Plausible  Value  Plausible  Values 


Grade /Ape 

Sample 

Mean 

S.  D. 

Mean 

S,  D, 

4/9 

Focused-BIB 

-.79 

.62 

-.79 

.62 

Intercorrelation 

-.83 

.64 

-.84 

.64 

VI 3 

Focused-BIB 

.12 

.72 

.12 

.72 

Intercorrelation 

.16 

.69 

.17 

.68 

12/17 

Focused-BIB 

.84 

.76 

.84 

.76 

Intercorrelation 

.75 

.78 

.78 

.77 

The  codings  of  the  civics-specific  conditioning  variables  are  presented 
in  Appendix  C  in  Table  C-6.    Common  core  conditioning  variables  were  also 
used.     For  grade  12/age  17,  the  "modal  age,  >  modal  grade"  category  was 
deleted  from  the  age -by- grade  variable,  because  students  above  grade  12  were 
not  sampled.    The  estimatpd  conditioning  effects  for    he  six  samples  defined 
by  the  three  grade/age  groups  and  the  mail.  focused-BIB  and  intercorrela  ion 
samples  are  given  in  Appendix  C  in  Tables  C-22  through  C-27.     The  values  of 
the  conditioning  effects  are  expressed  in  the  metrics  of  original 
calibration  scale.     Definitions  of  derived  conditioning  variables  are  given  in 
Appendix  D  (see  also  section  12.1.6.) 


12.1.4  The  Final  Proficiency  Scale 

In  order  to  resolve  the  linear  indeterminacy  of  the  original  IRT 
calibration  scale,  an  overall  weighted  mean  of  250.5  and  weighted  standard 
deviation  of  50  across  all  students  in  the  aain  focused-BIB  samples  was 
selected*    These  values  for  overtoil  mean  and  standard  deviation  have  been  used 
for  other  NAEP  subject  arer.  scales.    Although  the  civics  proficiency  scale, 
because  of  this  selection,  is  seemingly  expressed  in  the  same  units  as  those 
other  NAEP  proficiency  scales,  it  is  not  appropriate  to  compare  civics 
proficiency  scores  with  scores  on  the  other  subject  area  scales.    Any  other 
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convenient  transformation  of  the  original  civics  plausible  values  could  have 
been  chosen,  so  there  is  no  link  in  the  construction  of  the  civics  proficiency 
scale  to  the  scales  of  any  other  subject  area. 


Each  grade/age  sample  was  given  equal  weight  when  calculating  the  slope 
and  intercept  of  the  transformation.    For  each  grade/age  of  the  tcain  focused- 
BIB  samples,  the  weighted  means  of  the  five  plausible  values  were  averaged 
(Ml,  M2,  and  M3) .    These  averages  were  then  averaged  across  the  three 
grade/age  grouj^..     This  overall  average,  M(calibrated)  ,  based  on  values  in  the 
original  BILOG  calibrated  scale,  was  transformed  to  the  value  250.5.  The 
geometric  mean  of  the  we.lghted  standard  deviations  for  the  five  plausible 
values  for  each  grade/age  group  was  found  and  squared  to  estiaate  the  variance 
for  each  grade/age  group  (V^,  V2,  and  V3) .    Using  an  analysis  of  variance 
approach,  these  variances  and  the  square  of  the  averages  of  the  five  weighted 
means  for  the  grade /age  groups  were  added  together  to  find  the  sum  of  squares 
of  the  plausible  values  divided  by  sample  size,  S  X^/N  (SS^,  SS2,  and  SS3) . 
For  instance,  for  grade /age  level  4/9, 


The  three  values  of  S  X^/N,  one  for  each  grade/age  group  (SS^,  SS2,  and  SS3) , 
were  averaged.    This  average  was  treated  as  the  estimate  for  S  X^/N  for  the 
overall  sample.    It  was  combined  with  M(calibrated)  to  calculate  the  overall 
standard  deviation,  SD(calibrated) .  Thus, 


SD(Calibrated)  - 


SSi  +  SS2  +  SS3 


+  [M(Calibrated)]- 


The  transf ormatior    tsed  to  generate  proficiency  scores  with  an  overall  mean 
and  standard  deviation  for  all  three  grade /age  groups  of  250.5  and  50, 
respectively,  is 


50 

e(proficiency)  -    sD(Calibrated)  "  ^(calibrated) 

50 


SD(calibrated) 


•  M(Calibrated)+250.5 


or 


8(proficiency)  -  slope  •  6(calibrated)  +  intercept. 


For  the  1988  civics  proficiency  scale,  the  slope  and  intercept  are  51«78 
and  2A7.64,  respectively.    This  transformation  was  applied  to  the  main 
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focused-BIB  and  intercorrelation  samples  for  all  three  grade/age  groups. 
Overall  suunmar*-  statistics  for  the  samples  are  presented  in  Table  12-7. 


Table  12-7 

Means  and  Standard  Deviations  on  the  Civics  Proficiency  Scale 


First  All  Five 

Plausible  Value  Plausible  Values 


Sample 

Mean 

s. 

P, 

Mean 

s, 

0, 

Focused-BIB 

206. 

9 

32 

1 

206. 

8 

32 

0 

Intercorrelation 

204. 

8 

33 

1 

204. 

3 

33 

1 

Focused-BIB 

253 

6 

37 

.2 

253. 

1 

37 

0 

Intercorrelation 

256 

1 

35 

6 

256. 

4 

35 

.2 

Focused-BIB 

291 

0 

39 

.3 

291. 

0 

39 

.3 

Intercorrelation 

288 

0 

40 

.1 

288. 

1 

39 

.9 

Grade /Age 
4/9 

8/13 

12/17 


Item  parameters  on  the  calibrating  scale  may  be  transformed  using  the 
intercept  and  slope  so  that  the  conditional  probability  of  correct  response 
given  a  proficiency  score  can  be  obtained.    They  are 

a(proficiency)  -  a(calibrated)/slope; 

b (proficiency)  -  slope  •  b(calibrated)  +  intercept;  and 
c (proficiency)  -  c (calibrated) . 


The  transformation  of  the  estimated  conditioning  effects  is 


r(proficiency)  -  slope  •  r(calibrated)  +  intercept. 


12.1.5  Anchoring  the  Points  on  the  Civics  Proficiency  Scale 

The  same  anchoring  techniques  used  for  the  1986  mathematicr  and  science 
scales  were  applied  to  the  1988  civics  cross -sectional  scale,  using  data  from 
the  main  focused-BIB  samples.     Four  levels  (200,  ?50,  300,  and  350)  vere 
selected  on  the  civics  scale  and  chosen  as  anchor  points.     A  subject- area 
committee  defined  each  proficiency  level,  describing  the  types  of  questions 
that  most  students  attaining  that  level  would  be  able  to  answer  correctly  and 
that  most  students  at  least  one  level  lower  would  answer  incorrectly.  The 
committee  selected  benchmark  items  that  exemplify  each  level  of  proficiency 
from  the  group  of  items  identified  in  the  anchoring  process. 
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Possible  exemplar  items  were  selected  by  identifying  items  for  which 
more  than  65  percent  of  the  students  at  a  certain  proficiency  level  (;..ie 
anchor  level  of  the  item)  correctly  answered  the  item  while  less  than  50 
percent  of  the  students  at  the  next  lower  proficiency  level  correctly  answered 
the  item.     In  addition,  items  were  selected  only  if  the  difference  between  the 
percent  correct  for  the  two  proficiency  levels  was  larger  than  30  and  the 
ntunber  of  students  answering  the  question  at  each  of  the  two  proficiency 
levels  was  larger  than  10.     Students  were  considered  to  be  at  a  certain 
proficiency  level  if  their  proficiency  score  was  within  12.5  points  of  the 
proficiency  level.    For  more  details  of  the  anchorf.ng  process,  see  Chapter  7 
and  Beaton  (1987a)  . 

Civics  educator?  vno  were  members  of  the  subject- area  ccmmittee  examined 
the  sets  of  identified  items  at  a  particular  anchor  level  as  well  as  items 
that  almost  met  the  anchoring  criteria  and  used  their  expert  judgment  to 
characterize  each  proficiency  level.    Their  goal  was  to  contrast  the  tasks  at 
a  particular  anchor  level  with  those  at  levels  just  above  and  below.  The 
characterizations  selected  by  the  civics/social  studies  experts  for  each 
anchor  level  are  listed  in  Table  12-8.     Exemplar  items  selected  by  the 
committee  are  given  in  The  1988  Civics  Report  Card. 

Table  12-8 
Civics  Anchor  Levels  and  Descriptions 

f^nct^or  Level  Description 

200  Recognizes  the  Existence  of  Civic  Life 

250  Understands  the  Nature  cf  Political  Institutions  and 

the  Relationship  Between  Citizen  and  Government 

300  Understands  Specific  Government  Structures  and 

Functions 

350  Understands  a  Variety  of  Political  Institutions  and 

Processes 


12.1.6  Derived  Background  Variables 

Derived  variables  based  upon  background  questions  were  usea  for  two 
purposes:     as  conditioning  variables,  or  as  reporting  variables  used  to  define 
subgroups.    Some  of  these  variables  are  common  to  all  the  subject  areas; 
others  are  specific  to  the  1983  civics  assessment.    Derived  variables  used  for 
conditioning  are  listed  in  Table  C-6  of  Appendix  C  and  defined  in  Appendix  D 
Derived  variables  used  for  reporting  are  defined  in  Appendix  G.  Variables 
derived  for  purposes  of  reporting  have  been  placed  in  the  data  sets  on  the 
public-use  data  tapes;  variables  derived  for  conditioning  purposes  were  merely 
calculated  in  the  analysis  process. 
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Scoring  the  Open-ended  Item 


As  indicated  earlier,  one  civics  block  presented  to  students  in  the 
grade  8/age  13  and  grade  12/age  17  main  focused-BIB  samples  contained  an  open- 
ended  item  with  two  parts,  the  first  part  asking  students  to  identify  the 
president  and  the  second  asking  them  about  presidential  responsibilities. 
These  two  parts  were  included  in  the  scaling  process,  but  were  also  analyzed 
separately.    Chapter  6.2  contains  the  interrater  reliabilities  frx  the  two 
parts  of  the  item  as  they  were  originally  scored.    The  right/wrong  scoring  of 
the  categories  of  responses  for  the  two  parts  of  item  P018200  (P018201  and 
P018202)  are  indicated  in  Table  12-9.    The  percent  agreement  for  the  raters, 
when  the  item  is  dichotomized,  are  given  in  Table  12-10. 


Table  12-9 
Dichotomous  Scoring  of  the  Two  Parts 
of  the  Open-ended  Civics  Itea  <P018200) 

First  Scale  fP018201^  Second  Scale  fP018202> 

Incorrect:  0  Incorrect:  0-2,  9 

Correct:      1  Correct:  3-4 


Table  12-10 

Percent  Agreement  for  the  Ratings  of  the  Two  Portions  of  the 
Dichotomously  Scored  Open-ended  Civics  Item 


Grade /Age 

Portion 

JL. 

Percent  Agreement 

8/13 

P018201 

623 

99.5 

P018202 

584 

93.0 

12/17 

P018201 

574 

99.7 

P018202 

556 

95.7 

17.,  1.8  Content  Grouping  of  the  Items 

Although  only  one  civics  proficiency  scale  was  developed,  subject-area 
experts  who      thored  the  report  requested  moan  percent- correct  values  for  sets 
of  items  related  in  content.    The  items  includea  In  these  sets  are  listed  in 
The  1988  Civics  Report  Card. 
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12.2    TREND  DATA  ANALYSIS 


For  the  1988  study  of  trends  in  civics  achievement,  students  were 
administered  items  used  in  the  citizenship/social  studies  assessments 
conducted  in  1976  and  1982^.    The  1976  assessment  included  samples  of  17 -year- 
olds  who  had  dropped  cut  of  school  or  graduated  prior  to  assessment;  these 
students  were  not  included  in  the  1988  trend  study.    The  bridge  assessment 
consisted  of  one  booklet  each  for  ages  13  and  17.    Because  there  were  few 
reusable  civics  items  from  the  previous  assessments  for  age  9,  a  trend  booklet 
could  not  be  compiled;  for  that  reason,  a  bridge  assessment  for  that  age  was 
not  feasible.    In  order  to  match  the  characteristics  of  the  previous 
assessments,  age-only  samples  of  students  were  defined  using  common  age 
definitions  (age  13  birth  dates  based  on  the  1974  calendar  year,  age  17  birth 
dates  ranging  from  October  1,  1970  to  September  30,  1971),  common  times  of 
testing  (13-year-olds  in  the  fall,  17-y  ar-olds  in  the  spring),  and  a  common 
mode  of  administration  (booklets  were  paced  with  an  audiotape).     The  modal 
grades  for  the  age  13  and  age  17  samples  were  8  and  11,  respectively. 

A  total  of  134  multiple-choice  cognitive  questions  were  administered  as 
trend  items — 35  at  age  13  only,  39  at  age  17  only,  and  60  at  both  ages.  Two 
open-ended  items  were  included — one  at  age  17  only,  and  one  at  both  ages. 
These  two  items  were  included  in  the  scaling  process  and  examined 
independently.    Sixteen  of  the  items  that  had  been  administered  as  trend  items 
were  deleted  from  the  analysis  (Table  12-11)  because  the  item  itself  or  the 
direc     ms  for  the  item  had  been  changed  significantly  in  the  item  review 
process — 4  at  age  13  only,  6  at  age  17  only,  and  6  at  both  ages. 


Table  12-11 

Items  Deleted  from  the  Civics  Trend  Analysis 
Because  of  '"ext  Changes 


Age 

Block 

Items 

13 

C9 

P000301, 

P000701, 

P002001, 

POlOOOl 

CIO 

P003901, 

P004801, 

P004301 

Cll 

P005701, 

P005901, 

P007101 

17 

C9 

P008801, 

P008901, 

POlOOOl, 

P000301,  P000701 

CIO 

P003901, 

P004101, 

P004801, 

P004301 

Cll 

P005701, 

P005901, 

P007101 

Each  student  at  a  particular  age  level  was  adminiscered  a  booklet 
containing  the  same  three  blocks  of  civics  cognitive  items  as  well  as  the 


^Exercises  from  the  1970  citizenship  assessment  were  also  administered; 
however,  the  sparsity  of  items  and  test  objectives  common  to  the  1970 
assessment  and  any  of  the  three  later  assessments  precluded  using  1970  data  in 
the  scaling  process  or  extrapolating  the  1970  mean  p-values  onto  the  trend 
scale . 
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block  of  civics  background  questions  common  to  the  civics  focused-BIB  sam^'as 
and  the  block  of  general  background  questions  common  to  all  the  BIB  samples. 
In  addition,  six  noncognitive  questions  were  included  in  one  of  the  cognitive 
blocks  for  age  17.    These  background  questions  were  used  for  reporting 
purposes  only  (not  for  conditioning) .    Although  the  cognitive  items  had  never 
been  administered  within  the  same  blocks  before,  they  were  presented  aurally 
using  a  tape  recorder  as  in  past  assessments,  limiting  the  response  time  fo. 
each  item.    Many  of  these  items  were  presented  in  the  cross-sectional 
assessment  also,  but  in  printed  form. 

The  trend  data  analysis  examined  data  from  three  points  in  time:  1976, 
1982,  and  1988.    A  three-population  univariate  scale  was  fit  to  the  trend  data 
for  these  three  assessraents  at  each  age  level.    Due  to  the  differences  in  age 
definition,  time  of  testing,  and  mode  of  administration  between  the  civics 
bridge  and  main  assessments,  no  link  can  be  made  between  the  civics  trend 
scale  and  the  civics  cross -sectional  scale.    The  sample  sxzes  and  number*'  of 
items  from  each  assessment  are  given  in  Table  12-12. 

Table  12-12 
Number  of  Items  and  Sample  Sizes 
for  the  Civics  Trend  Assessment 


Nximber  of  Items 


Assessment 
Age  Year 

Sample  Size* 

Multiple-choice  Open-ended 

13  1976 
1982 
1988 

19,952 
7,268 
1,938 

64  1 
48  0 
85  1 

Total 

29,158 

85  1 

17  (in-school)  1976 
1982 
1988 

17,866 
6,751 
1,786 

60  2 
48  0 
87  2 

Total 

26,403 

87  2 

Total  1976 
1982 
1988 

37,818 
14,019 
3,724 

85  2 
65  0 
120  2 

Total 

55,561 

120  2 

*  Not  oil  examinees  received 

all  items  in  1976  and  in 

1982. 

The  steps  in  the  civics  trend  analysis  are  documented  in  the  following 
sections.    As  for  the  cross- sectional  analysis,  the  first  step  was  to  gather 
item  and  block  information.    Next  the  trend  items  were  calibrated  and 
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plausible  values  were  generated  after  conditioning  oii  available  background 
variables.     Finally  the  age  13  and  age  17  scales  were  linked  and  placed  on  the 
final  civics  trend  proficiency  s-ale.    Derived  background  variables  and  the 
scoring  of  the  open-ended  items  are  referred  to  in  the  last  sections  of  the 
chapter. 


12.2.1  Item  Analysis 

Table  12-13  contains  the  number  of  items,  KR-20  reliability,  mean, 
standard  deviation,  and  mean  percent  correct  for  each  block,  as  well  as  the 
number  of  students  who  were  administered  the  block.    The  average  values  were 
calculated  using  examinee  weights  and  all  of  the  multiple-choice  items  in  the 
block,  including  those  that  were  not  scaled.    The  1988  item-level  statistics 
were  not  very  different  from  those  for  the  1976  and  1982  assessments.  The 
percent  of  examinees  not  reaching  items  in  the  bridge  blocks  was  almost  always 
zero  because  the  items  were  administered  with  a  tape-recording  to  pace 
response  time. 

Table  12-13 


Descriptive 

Statistics 

for  Civics 

1988  Trend 

Blocks 

Number  of 

Mean  Number 

Mean  Percent 

Block 

Items 

KR-20 

Correct 

S.  D. 

Correct 

N 

C9 

34 

.79 

22.8 

5.2 

.67 

1938 

CIO 

27 

.78 

15.2 

4.5 

.56 

1938 

Cll 

34 

.82 

21.9 

5.6 

.64 

1938 

C9 

33 

.81 

19.0 

5.3 

.58 

1786 

CIO 

32 

.86 

22.2 

S.8 

.70 

1786 

Cll 

34 

.89 

25.2 

6.2 

.74 

1786 

12.2.2  Estimation  of  Item  Parameters 

The  first  step  in  the  scpHng  process  was  the  estimation  of  item 
parameters  for  the  trend  items.    This  item  calibration  was  performed  using  the 
BILOG  program  separately  for  each  of  the  two  age  groups,  using  combined  data 
from  the  three  assessments  and  treating  each  assessment  sample  as  a  sample 
from  a  separate  subpopulation.    The  calibration  was  performed  on  a  subsample 
of  all  the  available  subjects,  resulting  in  approximate  i.y  500  examinees  in 
each  assessment  year  for  each  item.    The  responses  were  not  weighted  for  this 
part  of  the  analysis. 

Three  items  were  subsequently  dropped  from  the  age  13  trend  scale  and 
two  items  were  dropped  from  the  age  17  trend  scale  because  of  lack  of  fit  to 
the  IRT  model.    Most  of  these  items  had  nonmonotonic  empirical  item  response 
functions;  however,  one  had  different  item  response  functions  for  the  1988  and 
1976  data.    The  NAEP  item  numb'.rs  and  the  reason  ti  ese  items  were  excluded  are 
listed  in  Table  12-14. 
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Table  12-14 


Items  Delated  from  the  Civics  Trend  Analysis 
Due  to  Lack  of  Model  Fit 


Age 


Block 


Item 


Reason  for  Exclusion 


13 


C9 
Cll 


P002601 
P005804 
P007001 


Not  mono tonic 
Not  mono  tonic 
Not  mono  tonic 


17 


Cll 


P006001 
P007001 


Not  mono  tonic 
Discrepant  IRFs 


As  a  result  of  these  deletions  and  the  inclusion  of  the  open-ended 
items,  83  items  were  scaled  for  age  13  and  87  items  were  scaled  for  age  17.  A 
list  of  the  items  scaled  for  each  of  the  two  ages,  along  with  their  item 
parameter  estimates,  appears  in  Tables  F-4  and  F-5  in  Appendix  F. 


The  generation  of  plausible  values  was  conducted  independently  by  ago 
for  each  of  the  three  assessment  years.    Because  there  were  fewer  background 
variables  available  for  past  trend  studies,  fewer  conditioning  variables  were 
used  in  the  creation  of  the  plausible  values  on  the  trend  scale  than  on  the 
cross -sectional  scale.    The  final  student  (booklet  or  package)  weights  were 
used  for  the  1976  and  1982  data  in  this  part  of  the  analysis.    Appendix  C  give 
tho  codinj^s  for  the  conditioning  variables  (Table  C-7)  and  the  sstimated 
conditioning  effects  (Tables  C-28  through  C-33)  for  the  two  age  groups.  The 
estimated  condition5.ng  effects  in  the  tables  are  expressed  on  the  scale  of  the 
original  calibration. 

12.2 • 4  Linking  the  Age  13  and  Age  17  Trenc  Scales 

Using  the  Stocking-Lord  (1983)  equating  procedure,  implemented  in  the 
TBLT  program  (Stocking,  1986),  the  item  p-.rameters  for  the  age  13  trend  scale 
were  resettled  by  deriving  a  lirear  equating  function  based  on  the  items  ccjnon 
to  the  .ge  17  trend  scale.    ITiis  function  was  used  to  rescale  the  original 
parameter  estimates  for  all  of  the  age  13  trend  items  in  the  scale  to  an 
interim  scale.     The  equating  procedure  and  a  study  of  the  error  involved  in 
thf  procedure  are  described  in  Appendix  D  of  Expending  tne  Now  Design:  The 
NAEP  1985-86  Technical  Report  (Beaton,  1988).    The  equating  function  for  the 
age  13  civics  trend  data  is 


Generation  of  Plausible  Values 


e(interim  13)  -  .76  •  e(calibrated  13)  -  .96. 
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The  original  age  13  item  parameter  estimates  and  estimated  conditioning 
effects  can  be  transformed  to  the  new  interim  scale  using  th^  relationships 
given  at  the  end  of  section  12.1.4. 


12.2.4  The  Final  Proficiency  Scale 

Because  the  trend  and  cross-sectional  scales  were  not  linked,  the  linear 
indeterminacy  of  the  trend  scale  was  resolved  by  setting  the  m^an  and  standard 
deviation  of  the  1988  age  13  trend  scale  to  50  and  10,  respectively.    The  mean 
of  the  five  plaunible  value  means  on  the  age  13  interim  scale  was  found,  as 
well  as  the  geometric  mean  of  the  five  plausible  value  standard  deviations  on 
the  age  13  interim  scale  for  this  sample  of  students .    These  values  were  used 
as  the  untransformed  mean,  M(calibrated)  and  standard  deviation, 
SD(calibrated)  when  calculating  the  function  to  be  used  to  transform  all  of 
the  age  13  data  on  the  interim  scale  and  the  age  17  data  on  the  original  scale 
V:o  the  final  trend  scale.    The  transformation  is 


e(proficiency)  -  12.13  •  e(cali^  ated)  +  60.50. 


Overall  summary  statistics  for  the  trend  samples  are  given  in  Table  12-15.  As 
for  the  age  13  interim  data  transformation,  item  parameter  estimates  and 
estimated  conditioning  effects  could  be  transformed  from  the  age  13  interim 
scale  and  the  age  17  original  scale  to  the  final  trend  proficiency  scale  using 
the  relationships  given  at  the  end  of  section  12.1.4. 


Table  12-15 

Means  and  Standard  Deviations  on  the  Civics  Trend  Proficiency  Scale 

First  All  Five 

Plausible  Value  Plausible  Values 


Age 

Assessment 

Mean 

s, 

P, 

Mean 

s, 

D, 

13 

1976 

49.05 

9 

53 

49.08 

9 

51 

1982 

49.08 

9 

01 

/•9.12 

8 

99 

1988 

50.09 

10 

03 

50.00 

10 

.00 

17 

1976 

61.66 

12 

17 

61.70 

12 

.12 

1982 

61.21 

12 

56 

61.35 

12 

.61 

1988 

59.58 

12 

.45 

59.61 

12 

.31 

12.2.5  Derived  Backgrotmd  Variables 

In  the  trend  analysis,  all  derived  variables  based  upon  backgrourd 
questions  were  used  both  for  conditioning  and  in  reporting  (to  define 
subgroups).    Derived  conditioning  variables  are  described  in  Appendix  D; 
derived  reporting  variables  are  described  in  Appendix  G.    The  variables 


ERLC 


315 


derived  for  reporting  purposes  have  been  placed  in  the  data  sets  on  the 
public-use  data  tapes. 


12.2.6  Scoring  the  Open-ended  Items 

The  two  open-ended  items  in  the  bridge  assessment  were  scored  using  the 
guidelines  in  Table  12-16.    Interrater  reliabilities  for  the  two  items  as  they 
were  originally  scored  are  given  in  Chapter  6.    Percent  agreements  for  the 
dichotomized  items  are  listed  in  Table  12-17. 


Table  12-16 

Dichotomous  Scoring  of  the  Open-ended  Civics  Trend  Items 

P021001  P021101 

Correct :      1-6  Correct :  1-7 

Incorrect:  7-11  Incorrect:  8-9 


Table  12-17 

Percent  Agreement  for  the  Ratings  of  the  Dichotomously  Scored 
Open-ended  Civics  Trend  I teres 

Grade/Age              I rem               N  Percent  Agreement 

8/13                 P021101          203  97.0 

12/17               P021001          371  99.2 

P021101          370  98.9 
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hkTK  ANALYSIS  FOR  THE  U.S,  HISTORY  ASSESSMENT^ 


Norma  A.  Norris 
Educational  Testing  Service 


In  1988,  U,S,  history  items  were  administered  to  seven  samples  of 
students,  shown  in  Table  13-1  and  described  as  follows: 

Students  in  the  U.S.  history  focused-BIB  spiral  samples — 9[Main-His], 
13tMain-His] ,  and  17[Main-His] — received  three  blocks  of  U.S.  history  items. 
Their  responses  were  used  for  grade-level  cross-sectional  analyses.  These 
analyses  compared  the  levels  of  U.S.  history  achievement  for  various 
subgroups,  which  are  reported  in  The  U.S.  History  Report  Card:  The 
Achievement  of  Fourth-,  Eighth-,  and  Twelfth-grade  Students  in  1988  and  Trends 
from  1986  Co  1988  in  the  Factual  Knowledge  of  High- school  Juniors  (Hammack, 
Hartoonian,  Howe,  Jenkins,  Levstik;  MacDonald,  Mullis,  &  Owen,  1990). 

The  intercorrelation  samples,  9[Main-Int],  13[Main-Int] ,  and 
17[Main-Int] ,  were  also  included  in  the  grade-level  cross- sectional  analyses. 
These  samples  of  students  received  one  block  of  U.S.  history  items  and  provide 
a  way  to  compare  student  achievement  across  subject  areas. 

The  I  ridge  sample  of  grade  11/age  17  students  was  designed  to  measure 
trends  in  U.S.  history  achievement  between  the  1986  and  1988  U.S.  history 
assessments.    Students  were  administered  three  blocks  of  previously  assessed 
U.S.  history  items  according  to  the  procedures  used  in  the  1986  assessment. 
The  results  from  the  two  assessments  were  compared  and  reporte<l  in  The  U.S. 
History  Report  Card  cited  above. 


13.1    CROSS -SECTIONAL  DATA  ANALYSIS 

In  1988,  241  U.S.  history  items,  56  of  which  had  also  been  administered 
in  1986,  were  administered  to  the  focused-BIB/intercorrelation  samples.  Two 
of  these  items  were  open-ended,  one  at  grade  8/age  13  and  one  at  grade  12/age 
17,    The  rest  were  multiple-choice.    Scoring  procedures  for  the  two  ordinal 
scales  for  the  open-ended  items  are  described  in  Chapter  6.2.     For  purposes  of 
IRT  scaling,  the  ordinal  scales  were  dichotomized  using  the  rules  described  in 
Table  13-2. 


^The  contributions  of  Rebecca  Zwick,  Kentaro  Yamamoto,  and  Lynn  Jenkins 
to  this  chapter  are  gratefully  acknowledged. 
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Table  13-1 


NAEP  1988  U.S.  History  Samples 


Subject 

Booklet 

Time  of 

Age 

Modal 

Sample 

Sample  Code 

Sample  Tvoe 

Numbers 

Cohort 

Defn. 

Grade 

Size 

9[Maln-Hls] 

Main  Focused-BIB 

H 

15 

Grade 

4/age  9 

Winter , 

spring 

CY 

4 

2,664 

13[Main-Hls] 

Main  Focused-BIB 

H 

15-21 

Grade 

8/aF:Q  13 

/inter, 

spring 

CY 

8 

5,988 

17[Main-Hls] 

Main  Focused-BIB 

H 

15-21 

Grade 

12/age  17 

Winter , 

spring 

CY 

12 

5,780 

9[Main-Int] 

BIB  Intercorrelation 

R.C.H 

17-19 

Grade 

4/age  9 

Winter , 

spring 

CY 

4 

2,638 

13[Maln-Int] 

BIB  Intercorrelation 

R.C.H 

29-31 

Grade 

8/age  13 

Winter , 

spring 

CY 

8 

2,590 

17[Main-Int] 

BIB  Intercorrelation 

R.C.H.G 

30-31 

Grade 

12/age  17 

Winter , 

spring 

CY 

12 

1,632 

17[Br86-His] 

Bridge  to  1986 

H 

67 

Grade 

11/age  17 

Spring 

Not 

CY  11 

2,349 

R  -  Reading 

C  -  Civics 

H  -  U,S,  history 

G  Geography 


CY  -        Calendar  year:  birth  dates  in  1978,  1974, 
and  1970  for  ages  9,  13,  and  17 

Not  CY  -  (age  17  only):  birth  dates  between  October 
1,  1970  and  September  30,  1971 


3:^6 
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Tabl^  13-2 
Dichotomized  Variables  for  U.S. 


History 


Grade  8/Age  13 

H024901    Settlers  in  America  1  or  2  -  0  (wrong) 

3,  4,  or  5  -  1  (right) 

Grade  12/Age  17 

H025003    Presidential  Power  0,  1,  or  2  -  0  (wrong) 

3  or  4  -  1  (right) 


Preliminary  item  analyses  were  conducted  separately  for  each  of  the 
grade/age  samples  to  check  the  validity  and  reliability  of  the  items.  For 
each  item,  the  percent  of  students  selecting  each  response,  the  percent  who 
ouitted  the  item,  the  percent  who  failed  to  reach  the  item,  and  the 
correlation  between  item  score  and  the  block  score  were  calculated.    Also,  for 
each  block,  the  internal  consistency  reliability  coefficient  was  computed.  No 
items  were  excluded  at  this  stage  based  on  the  results  of  these  analyses. 

Table  13-3  shows,  for  each  block  of  multiple-choice  items  in  both  the 
focused-BIB  and  intercorrelation  samples,  the  number  of  items,  the  mean 
proportion  correct,  the  KR-20  reliability  coefficient,  and  the  number  of 
students  who  responded  to  at  least  one  item  in  the  block. 

The  following  table  shows,  for  each  block  of  multiple-choice  U.S. 
history  items,  the  range  of  percents  of  items  in  the  L    ck  that  were  not 
reached  by  students. 

Table  13-4 
Range  of  Percents  of  Items  Not  Reached 
for  Multiple-choice  U.S.  History  Blocks 


Block 

Grade  4 

Grade  8 

Grade  12 

H2 

.1  -  1.4  X 

.9  -  2.8  Z 

.4  -  9.7  X 

H3 

.3  -  3.7  X 

.6  -  3.9  X 

.6  -  3.1  X 

H4 

.9  -  3.6  X 

.5  -  1.8  X 

.4  -  2.1  X 

H5 

.5  -  3.1  X 

.9  -  3.0  X 

H6 

.6  -  5.5  X 

.6  -  4.4  X 

H7 

.4  -  8.3  X 

.8  -  7.7  X 

13.1.1  Scaling 

In  1988,  237  U.S.  history  items  were  scaled  and  responses  to  the  history 
items  were  summarized  in  a  single  U.S.  history  proficiency  scale.    Table  13-5 
shows  the  number  of  items  scaled  for  each  of  the  samples.    Tho  history  scale 


Table  13-3 

Descriptive  Statistics  for  Multiple -choice  U.S.  History  Blocks 


Focused-BIB  Sample   


—  Intercorrelation  Sample 


Grade /Age 
Grade  4/Age  9 

Grade  8/Age  13 


Grade  12/Age  17 


!!ean 

Mean 

Nvusber 

Proportion 

Proportion 

Correct 

li 

Correct 

KR-2P 

li 

H2 

15 

.63 

.73 

2660 

.64 

.74 

866 

H3 

15 

.52 

.71 

2660 

.53 

.70 

880 

H4 

15 

.56 

.72 

2660 

.57 

.69 

861 

H2 

26 

.75 

.86 

2534 

.75 

.87 

858 

H3 

26 

.65 

.82 

2560 

— 

— 

— 

H4 

26 

.64 

.81 

2538 

H5 

26 

.44 

.69 

2543 

.43 

.67 

854 

H6 

28 

.54 

.78 

2574 

.57 

.76 

858 

H7 

28 

.44 

.75 

2552 

H2 

28 

.54 

.82 

2458 

.57 

.81 

815 

H3 

25 

.56 

.80 

2458 

K4 

25 

.54 

.81 

2461 

H5 

26 

.56 

.79 

2459 

H6 

28 

.6« 

.85 

2463 

.72 

.85 

810 

H7 

28 

.61 

.85 

2459 

339 


340 


ERIC 


320 


was  detiv^id  using  the  item  response  theory  (IRT)  methodology  extensively 
documented  in  Chapter  9  of  this  report.    Only  a  brief  outline  of  the  scaling 
procedures  is  given  here. 


Table  13-5 

U*S.  History  Profxciency  Scale  Item  Information,  by  Sample 

No,  of  History 
Sample  Code  Cohort  Scale  Items 

9[Main-His]  and  9[Main-Int]  Grade  4/age  9  44 
IslMain-His]  and  13[Main-Int]  Grade  8/age  13  158 
17[Main-His]  and  17[Main-Int]  Grade  12/age  17  159 
17[Br86-His]  Grade  11/age  17  0  (Only  item 

percents  correct 
were  reported) 

Three  U.S.  history  items  were  excluded  from  tho  U.S.  history  proficiency 
scale  because  of  lack  of  fit  to  the  IRT  model.    On  these  items,  two  grade/age 
samples  had  different  empirical  response  curves.     In  one  case  (item  H003401) , 
one  grade/age  sample  had  an  item  response  curve  that  was  not  monotonically 
increasing.    Table  13-6  identifies  the  U.S.  history  items  excluded  from  the 
scale.    Only  the  first  rater's  (dichotomized)  scores  on  the  two  open-ended 
items  (H024901  and  H025003)  were  used  in  scaling.    The  second  rater's  scores 
were  used  only  for  estimation  of  interrater  reliability. 

Table  13-6 

U.S.  History  Item  Scores  Excluded  from  IRT  Scale 

NAEP  ID  Description  Grade /Af:.., 

H003401       Emancipa'^lon  Proclamation  8/13.  12/17 

H012601       American  Revolution  V^,  8/13 

H022401       Association  with  F.  D.  Roosevelt  8/13,  12/  7 


Item  Calibration 

For  the  combined  focused  BIB/intercorrelation  samples  across  all  three 
grade/age  levels,  the  BILGG  program  (Mislevy  &  Bock.  1982)  was  used  to  obtain 
item  parameter  estimates  on  a  provisional  scale  based  on  the  three -parameter 
logistic  model.     Parameters  were  estimated  even  for  previously  administered 
items;  larameter  values  were  not  assumed  to  be  equal  to  their  1986  values. 
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Conditioning 


Before  proficiency  estimation  was  initiated,  conditioning  variables  were 
derived  for  each  respondent.    A  common  set  of  conditioning  variables  was  used 
in  all  subject  areas  in  the  1988  assessment,  along  with  background  items 
specific  to  the  subject  area  being  scaled.    The  common  set  of  conditioning 
variables  is  listed  in  T3iu  ^-1  of  Appendix  C.    The  specific  history 
background  items  that  were  used  as  conditioning  variables  in  the  U.S.  history 
scaling  and  reporting  are  listed  in  Table  C-8  of  Appendix  C.     Each  of  the  U.S. 
history  grade/age  samples  for  which  proficiency  values  were  estimated  were 
conditioned  using  the  common  set  of  conditioning  variables  (common  across  all 
subjects  areas  within  a  given  grade/age  sample)  and  the  grade/age  background 
items  specific  to  the  respective  U.S.  history  sample.    The  one  exception  was 
the  sample  at  the  grade  12/age  17  level,  where  the  category  "modal  age  >  moda- 
grade"    was  not  used,  inasmuch  as  no  one  in  this  sample  can  be  enrolled  in  a 
grade  above  the  modal  grade  (12).    Tables  C-34,  C-35,  and  ^-36  of  Appendix  C 
list  the  estimated  effects  for  the  common  conditioning  variables  and  the  U.S. 
history  attitude  items  that  were  obtained  from  the  M*CROUP  program  for  %jach  of 
the  grade/age  samples. 


Proficiency  Estimation 

Using  the  iterative  method  described  by  Mislevy  (1985)  and  implemented 
in  the  M-GROUP  program  (Sheehan,  1*35),  a  U.S.  history  proficiercy 
distribution  was  estimated  for  each  respondent.    The  M  GxlOUP  program  was 
applied  to  each  grade/age  separately,  and  from  each  respondent's  distribution 
five  random  values  were  drawn  ani'  saved.    These  "plausible  values"  were 
transformed  by  standardizing  the  combined  distribution  for  the  three 
grade/ages  to  a  mean  of  250.5  and  a  standard  deviation  of  50.0.    The  five 
plausible  values  for  each  respondent  were  then  used  f<  r  estimating  demographic 
group  statistics. 


13.1*2  Anchoring 

The  U.S.  history  p..oficiency  scale  was  "anchored"  using  the  same 
procedures  described  in  Chapter  7.     Based  on  the  ran^e  of  student  performance 
in  the  assessment,  the  following  levels  of  U.S.  history  proficiency  were 
defined: 

•  Level  200:    Knows  Simple  Historical  Facts 

•  Level  250:    Knows  Beginning  Historical  Inform  ition  and  has 

Rudimentary  Interpretive  Skills 

»         Level  300:    Understands  Basic  Historical  Terms  and  P  ationships 

•  Level  350:    Interprets  Historical  Information  and  Ideas 

To  provide  the  basis  for  the  anchoring  process,  NAEP  used  empirical 
procedures  to  delineate  sets  of  Items  that  discriminat*jd  between  adjacent 
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performance  levels  on  the  scale — that  is,  items  likely  to  be  answered 
correctly  by  students  performing  at  a  particular  level  on  the  scale  and  much 
less  likely  to  be  answered  correctly  by  students  performing  at  the  next  lower 
level. 

The  sets  of  items  represented  at  each  of  the  four  levels  were  studied  by 
a  panel  of  distinguished  history  educators,  who  carefully  considered  and 
articulated  the  tjrpes  of  knowledge,  skills,  and  reasoning  abilities 
demonstrated  by  correct  responses  to  the  items  in  each  set.    This  information 
was  then  placed  in  the  context  of  the  assessment  framework  and  used  to 
characterize  students'  understanding  of  the  significant  events,  people,  and 
policies  that  have  shaped  our  nation's  history,  as  well  as  their  ability  to 
interpret  and  analyze  historical  information. 

Table  13-7  shows  the  percentages  of  students  in  each  grade  who  per  ^ormed 
at  or  above  each  level  of  U.S.  history  proficiency. 


Table  13-7 
U.S.  History  Scale  Anchoring: 
Percentages  At  or  Above  Each  Level  (with  Standard  Errors) ,  by  Grade 


Level 

Grade  4 

Grade  8 

Grade  12 

200 

76.0  (1.0) 

96.0  (0.3) 

99.4  (0.0) 

250 

15.9  (0.9) 

67.7  (0.9) 

88.9  (0.6) 

300 

0.2  (0.1) 

12.7  (0.5) 

45.9  (1.3) 

350 

0.0  (0.0) 

0.1  (0.0) 

4.6  (0.5) 

13 . 1 . 3  Classification 

The  primary  objectives  of  the  1988  U.S.  history  assessment  were  to 
measure  students'  knowledge  of  the  significant  events,  persons,  and  documents 
that  have  shaped  our  nation's  history,  and  to  assess  their  understanding  of 
how  these  fit  into  the  major  chronological  time  periods  of  oar  past.  To 
address  these  objectives,  the  U.S.  history  items  were  clasr>ified  as  follows: 


Context 

•  Political  Life 

•  Economic  Life 

o  Cultural,  Social,  and  Family  Life 
Chronology  of  Events 

•  Chronology 

•  Documents 

•  Persons 
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Historical  Periods 


•  Exploration  and  Colonization,  up  to  1763 

•  The  Revolutionary  Era,  the  Constitution,  and  the  New  Republic, 
1763-1815 

•  Economic  and  Social  Development  of  the  Antebellum  Republic, 
1790-1861 

•  Crisis  of  the  Union:  Origins  of  the  War,  the  War,  and 
Reconstruction,  1850-1877 

•  The  Rise  of  Modern  America  and  World  War  I,  1877-1920 

•  The  United  States,  1920-1941 

•  World  War  II  and  the  Postwar  Era,  1931-1968 

•  Modern  Post- irdvis trial  Era,  1968  to  the  present 

Based  on  these  classifications,  statistical  analyses  were  performed  for 
different  subgroups  (e.g.,  male,  female)  in  each  of  the  grade/age  s^^mples.  The 
means  and  standard  errors  were  generated  at  the  item  level,  and  across  the 
group  of  items  within  each  of  the  classification  categories.  Differences 
between  the  subgroups  within  each  of  the  categories  also  were  compared. 

13.2    TREND  ANALYSES 

To  provide  information  about  trends  in  students'  knowledge  of  U.S. 
history,  105  U.S.  history  items  were  re-administered  in  1988  to  a  bridge 
sample  of  grade  11/age  17  students.    The  1988  sample  of  grade  11  students  was 
compared  to  the  equivalent  sample  from  the  1986  assessment. 

The  U.S.  history  trend  items  were  not  scaled.     Given  time  and  cost 
constraints,  it  did  not  seem  beneficial  to  do  so,  because  the  only  previous 
U.S.  history  assessment  was  from  the  1986  sample  of  grade  11/age  17  students 
and  the  modal  grade  of  11  is  no  longer  being  assessed. 

Based  on  preliminary  item  analyses  in  which  the  1988  results  were 
compared  with  the  results  for  the  1986  items,  the  item  statistics  for  the  1988 
and  the  1986  samples  were  found  to  be  comparable. 

Using  the  item  classifications  previously  mentioned  for  the  cross - 
sectional  analyses,  the  means  and  standard  errors  were  generated  for  both  the 
1986  and  1988  samples.    The  results  of  the  U.S.  history  trend  analyses  are 
reported  in  The  U.S.  History  Report  Card:    The  Achievement  of  Fourth-, 
Eighth",  and  Twelfth- grade  Students  in  1988  and  Trends  from  1986  to  1988  in 
the  Factual  Knowledge  of  High- school  Juniors  (Hammack,  Hartoonian,  Howe, 
Jenkins,  Levstik,  MacDonald,  Mullis,  &  Owen,  1990). 
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Chapter  14 

DATA  ANALYSIS  FOR  THE  GEOGRAPHY  ASSESSMENT^ 


Norma  A.  Norris 
Educational  Testing  Service 


In  1988,  geography  items  were  administered  to  two  samples  of  students, 
as  shown  in  Table  14-1. 

Students  iu  the  fncused-BIB  sample  for  geography,  17[Main-Geo]  received 
three  blocks  of  geography  items.    Students  in  the  intercorrelation  sample  for 
geography,  17[Main-Int]  ,  received  a  block  of  geography  items,  a  block  of 
reading  items,  and  a  block  of  either  U.S.  history  or  civics  items. 

These  two  samples  were  analyzed  as  a  single  population  in  the  cross- 
sectional  analyses,  in  which  the  levels  of  geography  achievement  fo--  various 
subgroups  were  compared.     The  results  of  these  analyses  are  reported  in  The 
Geography  Learning  of  High-School  Seniors  (Allen,  Bettis,  Kurfman,  MacDonald, 
Mullis,  &  Salter,  1990). 


14.1    CROSS-SECTIONAL  DATA  ANALYSES 

In  total,  78  geography  items  were  administered  in  1988  to  the  focused- 
BIB/inr.ercorrelation  sample.    All  of  these  items  were  multiple- choice. 

Preliminary  item  analyses  were  conducted  to  check  the  validity  and 
reliability  of  the  items.  For  each  item,  the  percent  of  students  selecting 
each  response,  the  percent  who  omitted  the  item,  the  percent  ,.ao  did  not  reach 
the  item,  and  the  correlation  between  the  item  score  and  the  block  score  were 
calculated.    Also,  for  each  block,  the  internal  consistency  reliability  was 
computed.     No  items  were  excluded  based  on  the  results  of  thsse  analyses. 

Table  14«2  shows,  for  each  block  in  both  the  focused-313  and 
intercorrelation  samples,  the  number  of  items,  the  mean  proportion  correct, 
the  k?c-20  reliability  coefficient,  and  th«.  number  of  12/age  17  studencs  w..^ 
responded  to  at  least  one  item  in  the  block. 

Percents  of  students  not  reaching  items  ranged  from  .2  to  8.2  percent 
for  items  in  block  G2,  from  .5  to  9.9  percent  for  items  in  block  G3,  and  from 
1.1  to  9.0  percent  for  items  in  block  G4. 


^The  contributions  of  Kentaro  Yamamoto,  Rebecca  Zwick,  and  Lynn  Jenkins 
to  this  chapter  are  gratefully  acknov/ledged. 
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NAEP  1988  Geography  Samples 


Subject 
^^^^^ 


Booklet 
Numbers 


Sample  Codct        Sample  Type 
17(Main-Geo]    Focused-BIB  Main     G  29 
17[Hain-Int]    BIB  Intercorr.         R.C.H.G    30,  32 


R  -  Reading 

C  -  Civics 

H  -     U.S.  history 


Time  of 
Cohort  Assessment 

Grade  12/age  17    Winter,  spring 

Grade  12/age  17    Winter,  spring 


Age 

ma. 

CY 
CY 


Modal 
Grade 

11 
12 


G  -  Geography 

CY  -    Calendar  year:    birth  dates  in  1978,  1974,  and  1970 
respectively  for  ages  9,  13,  and  17 


Sample 

2,446 
1,623 


Table  14-2 

Descriptive  Statistics  for  Geography  Blocks 


Focused-BIB  Sample 


Intercorrelation  Sample 


G2 
G3 
G4 


Number 
of  Items 

26 
26 
26 


Mean 
Proportion 
Correct 

.57 
.54 
.56 


KR-20 

.79 
.79 
.84 


N 

2438 
2438 
2438 


Mean 
Proportion 
Correct 

.56 

.57 


KR-20 
.79 
.83 


13 
803 
805 


3-17 


14.1,1  Scaling 


In  1980,  the  responses  to  the  78  geography  items  were  summarized  on  a 
single  geography  profici  n  y  scale,  derived  using  the  item  response  theory 
(IRT)  methodology  extensively  documented  in  Chapter  9  of  this  report.    Only  a 
brief  outline  of  the  scaling  procedures  is  given  here. 


Item  Calibration 

For  the  grade/age  sample,  the  BILOG  program  (Mislevy  &  Bock,  1982)  was 
used  to  obtain  item  parameter  estimates  on  a  provisional  scale,  based  on  the 
three-parameter  logistic  model. 


Conditioning 

Before  proficiency  estimation  was  initiated,  conditioning  variables  were 
derived  for  each  respondent.    A  common  set  of  conditioning  variables  was  used 
in  all  subject  areas  in  the  1988  assessment,  along  with  background  items 
specific  to  the  subject  area  being  scaled.    The  common  set  of  conditioning 
variables  is  listed  in  Table  C-1  of  Appendix  C.    The  geography-specific 
background  items  that  were  used  as  conditioning  variables  in  the  geography 
scaling  and  reporting  are  listed  in  Table  C-9  of  Appendix  C.    The  one 
exception  was  the  sample  at  the  grade  12/age  17  level,  where  the  category 
"modal  age,  >  modal  grade"    was  not  used,  inasmuch  as  no  one  in  that  sample 
can  be  enrolled  in  a  grade  above  the  modal  grade  (12).     Table  C-37  of  Appendix 
C  lists  the  estimated  effects  for  the  common  conditioning  variables  and  the 
geography  back^jr^^und  items  that  were  obtained  from  the  M- GROUP  program. 


Pisrcficiency  Estimation 

Using  Che  iterative  method  described  by  Mislevy  (19£5)  and  implemented 
in  the  M-GROUP  program  (Sheehan,  1985),  a  geography  proficiency  distribution 
was  estimated  for  each  respondent,  and  from  each  respondent's  distribution 
five  random  values  were  drawn  and  saved.    These  "plausible  values"  were 
transformed  by  standardizing  to  a  mean  of  285.0  and  a  standard  deviation 
40.0.     (This  mean  and  standard  deviation  were  chosen  to  be  similar  to  those 
obtained  for  grade  11  reading  proficiency  in  1984.    The  same  .standardization 
was  used  ir  the  U.S.  history  and  literature  proficiency  c^ales  in  1986.)  The 
five  plausible  values  for  each  respondent  were  then  used  for  estimating 
demographic  group  statistics. 


14.1.2  Classification 


The  objectives  for  NAEP's  1988  geography  assessment  reflected  a  broad- 
based  consensus  of  university  professors,  classroom  teachers,  social  science 
researchers,  school  administrators,  and  curriculum  specialists.     In  brief,  the 
assessment  framework  emphasized  that  students  should  be  able  to  use      3  skills 
and  tools  of  geogiaphy,  including  maps,  charts,  and  globes;  that  they  should 
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know  and  tinders tand  the  concepts  underlying  cultural  and  physical  geography, 
including  the  locations  of  places,  resources*,  and  cultural  areas;  and  that 
they  should  be  able  to  apply  geography  principles.    To    ddress  these 
objectives,  the  geography  items  were  classified  as  follows: 

•  Knowing  locations 

•  Using  the  skills  and  tools  of  geography 

•  Understanding  cultural  geography 

•  Understanding  physical  geography 

Based  upon  uhese  classifications,  statistical  analyses  were  performed 
for  different  subgroups  (e.g.,  male,  female).    The  means  and  standard  errors 
were  generated  at  the  item  level  and  across  the  group  of  items  within  each  of 
the  classifications.    Differences  between  the  subgroups  within  each  of  the 
classifications  also  were  compared. 
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Chapter  15 

DATA  ANALYSIS  FOR  MATHEMATICS  AND  SCIENCE^ 


Kentaro  Yamamoto 
Educational  Testing  Service 


The  mathematics  and  science  items  were  part  of  the  samples  used  in  3  988 
to  bridge  back  to  the  1986  trend  samples.    This  chapter  describes  the 
technical  details  of  the  item-parameter  estimation  and  ocaling  that  were 
perfoxnned  for  trend  analyses  of  responses  to  mathematics  and  science  cognitive 
items  in  the  1988  assessment.    The  results  of  the  analyses  are  presented  in 
greater  detail  in  The  Effect  of  Changes  in  the  National  Assessment: 
Disentangling  the  NAEP  1985-86  Reading  Anomaly  (Beaton  &  Zwick,  1990) 

To  maintain  the  comparability  of  measurement  instruments,  booklets  for 
the  1988  reading  bridge  to  1986  were  identical  to  those  used  in  1986  and 
therefore  included  s>cience  and  mathematics  blocks.    The  1988  mathematics  and 
bcience  trend  analyses  are  limited  to  data  from  blocks  that  appeared  in  the 
same  booklets  as  the  reading  blocks  in  the  1986  assessment.     For  age  17,  the 
number  of  mathematics  and  science  blocks  available  for  trend  analysis  was 
smaller  in  1988  than  in  1986.    However,  since  every  1986  trend  booklet  for 
ages  9  and  13  contained  a  block  from  each  of  the  three  subject  areas,  the 
complete  sets  of  trend  blocks  that  were  available  in  1986  for  both  ages  were 
also  available  for  analysis  in  1988. 

The  combination  of  blocks  within  booklets,  the  composition  of  item 
blocks,  the  mode  of  administration,  the  sample  definition,  and  the  time  of 
testing  were  identical  for  the  age  9  and  age  13  samples  in  the  1986  assessment 
and  the  1988  bridge  to  1986.    Consequently,  trend  analyses  for  these  two  ages 
were  straightforward;  trend  analyses  for  age  17,  however,  were  not. 

In  1986,  the  reading  trend  for  age  17  was  assessed  as  part  of  the  BIB 
spiral  portion  of  the  assessment,  while  the  science  and  mathematics  trends 
were  assessed  apart  from  reading  under  a  paced- tape  mode  of  administration 
Sinc<3  the  overarching  aim  of  the  1988  bridge  study  was  to  replicate  the 
booklets  and  administration  procedures  for  the  1986  assessment  of  trends  in 
reading,  booklets  from  the  BIB  spiral  portion  of  the  1986  assessment  were 
again  administered  in  1988  under  the  same  administration  conditions  as  in 
1986.     In  particular,  the  administration  of  mathematics  and  science  items  in 
the  spiral  portion  made  use  of  paper  and  pencil,  rather  than  paced  tape.  An 
implication  of  this  mode  of  administration  was  that  the  data  from  the  1988  age 


^Maxine  Kingston,  Edward  Kulick,  Michael  Narcowich,  and  Minh^ei  Wang 
performed  the  data  analyses  for  this  chapter;  Edward  Kulick  produced  the 
figure.    Robert  Mislevy  provided  consultation  on  scaling  and  Rebecca  Zwick 
provided  valuable  editorial  assistance. 
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17  trend  assessments  of  mathematics  and  science  are  comparable  to  the  1986  BIB 
assessikent,  but  not  directly  to  the  1986  trend  assessment.    As  a  result,  the 
design  to  align  the  1988  trend  point  for  age  17  student  to  the  past  trend  was 
more  complicated  than  before.     For  age  17  in  1988,  two  types  of  equa' In^  ^ere 
necessary — one  based  on  common  populations  across  different  modes  of 
administration  for  the  1986  BIB  and  trend,  and  one  based  on  common  items 
(similarly  placed)  for  tne  1986  BIB  an«i  the  1988  trend. 

The  main  objective  of  the  1988  trend  assessments  of  mathematics  and 
science  was  to  evaluate  the  differences  between  the  1986  and  1988  assessments. 
The  1988  trend  point  was  to  be  added  to  the  existing  trend  line.     Since  these 
analyses  closely  follow  those  conducted  in  1986,  readers  desiring  more 
detailed  descriptions  are  referiTed  to  relevant  chapters  in  Expanding  the  New 
Design:    The  NAEP  1985-86  Technical  Report  (Beaton,  1988).     This  chapter  will 
consider  details  specific  to  the  1988  analysis. 

15.1    SAMPLING  OF  STUDENTS  AND  ITEMS  FOR  MATHEMATICS  AND  SCIENCE 

For  ages  9  and  13,  the  combination  of  blocks,  composition  of  item 
blocks,  mode  of  administration,  age  definition,  and  time  of  testing  in  1986 
were  identical  to  those  in  the  1988  bridge  to  1986.    Three  booklets,  identical 
to  those  used  in  1986  including  background  questions,  were  used  to  measure 
trend  for  these  ages.     Each  booklet  contained  one  i.ading,  one  mathematics, 
and  one  science  block.     Each  student  in  the  sample  was  administered  one  of 
these  booklets.    The  mathematics  and  science  portions  were  presented  aurally 
using  a  tape  recorder  as  in  past  assessments.     The  tape  recorder  was  turned 
off  for  the  reading  block. 

For  age  17,  the  mathematics  and  science  booklets  of  the  1986  trend 
assessment  were  not  used  in  1988,  since  the  1986  mathematics  and  science  trend 
booklets  for  age  17  did  not  include  reading  blocks.     Instead,  the  booklets 
used  in  1988  were  identical  to  a  subset  of  booklets  used  for  the  1986  BIB 
assessment  and  consisted  cf  six  booklets,  five  of  which  contained  at  least  one 
reading  block  and  either  a  mathematics  or  a  science  trend  block  from  the  1986 
assessment.    The  sixth  booklet,  which  did  not  contain  mathematics  or  science 
blocks,  was  included  only  for  the  reading  assessment  in  1988.    Three  trend 
blocks  that  appeared  in  the  1986  age  17  trend  assessments  were  used  for  the 
1988  age  17  trend  assessment,  even  though  some  booklets  included  additional 
mathemo  .cs  and  science  blocks  used  only  for  the  cross -sectional  assessment  in 
1986.     Only  one  of  the  first  two  trend  blocks  of  either  mathematics  or  science 
was  included  in  four  of  the  booklets;  the  fifth  booklet  contained  both  a 
mathematics  and  a  science  block  (details  of  block  arrangements  are  documenttd 
in  Chapter  4) .     The  1988  age  17  sample  was  defined  using  the  same  age 
definition  as  the  1986  BIB  assessment  and  received  a  print -administered 
assessment  instead  of  the  paced  administration  of  the  pre -1988  trend 
assessments.    Unlike  the  samples  at  ages  9  and  13,  in  which  every  student 
received  both  a  mathematics  and  a  science  block,  about  one- fifth  of  the  age  17 
sample  received  both;  the  rest  received  a  block  of  either  mathematics  or 
science  items. 
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The  proficiencies  of  the  three  ages  cannot  be  placed  on  a  single  scale 
without  a  cross-sectional  study  or  a  vertical  equating  across  ages,  neither  of 
which  Vf.re  possible  in  the  1988  mathematics  and  science  trend  assessment.  The 
mathematics  and  science  scales  were  derived  from  the  1986  cross-sectional 
assessment  (see  E.  G.  Johnson,  1988,  and  Yamamoto,  1988).     The  1988  trend 
analysis  added  a  new  trend  point  to  the  existing  trend  line  up  to  1986. 

The  specific  mathematics  and  science  samples  for  1988  and  1986  are  shown 
in  Table  15-1. 

The  items  used  for  the  analysis  of  the  1988  data  set  are  the  same  as 
those  used  for  the  1986  trend  analyses;  that  is,  the  same  items  were  excluded 
as  in  1986  for  reasons  of  lack  of  fit  of  the  estimated  item  response  function 
to  the  empirical  regression  curve  in  either  the  bridge  or  the  cross-sectional 
data.    Three  nathematics  items,  one  from  each  age  group,  were  excluded  from 
the  scaling.     The  calculator  items  were  excluded  from  the  analysis.  Seven 
science  items  were  dropped  from  the  scaling  for  age  9,  seven  were  dropped  for 
age  13,  and  four  were  dropped  for  age  17. 

Using  current  methods,  it  is  possible  to  assess  the  change  over  tr.ne  in 
either  item  characteristics  or  proficiencies  of  populations,  but  not  both  at 
the  same  time.    This  is  true  for  any  analysis,  whether  based  on  dassical  test 
theory,  item  response  theory,  or  proportions  correct.      To  assess  change  in 
item  characteristics,  we  are  forced  to  assume  that  the  ability  distribution  of 
the  population  remains  stable;  to  assess  change  in  the  ability  distribution  of 
the  population,  we  must  assxinie  that  the  item  characteristics  are  stable  (see 
the  disct^ssion  of  common-item  equating  in  Chapter  10).    However,  we  know  that 
these  assumptions  are  not  strictly  justified.     Societal  and  instructional 
changes  may  produce  gradual  alterations  in  item  functioning  over  time.  If 
there  is  evidence  that  this  is  occurring,  it  may       desirable  to  allow  for 
changes  in  the  parameters  of  these  common  items.    Permitting  item 
characteristics  to  vary  in  this  way  is  feasible  only  if  common-population 
equating  methods  are  available  to  link  the  newly  obtained  results  to  past 
trend  lines.     This  is  the  approach  that  was  used  in  analyzing  the  1988 
mathematics  data  at  age  17  and  science  data  at  all  three  ages. 


15.2    SCALING  OF  THE  MATHEMATICS  TREND  DATA 

Mathematics  Trend  Scaling  for  Ages  9  and  13 

From  the  item  analysis,  it  was  found  that  the  1988  response 
distributions  of  all  response  choices,  including  "omitf,"  were  quite  similar 
to  the  1986  data.    The  mean  weighted  proportion  correct  at  the  block  level  was 
computed;  these  values  were  compared  with  the  1986  results,  as  shown  in  Table 
15-2.    At  each  block  level  for  all  age  groups,  the  1988  sample  showed  higher 
weighted  proportion  correct  vclues  than  the  1986  sample. 

In  estimating  itera  parameters  in  1986,  combined  data  from  the  three  most 
recent  trend  assessments  (1977,  1982,  and  1986)  were  used.    Thus,  the  1986 
trend  analysis  assumed  the  characteristics  of  all  items  were  stable  across  the 
three  assessments.     Item  parameters  estimated  in  1986  were  kept  unchanged  for 
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Table  15-1 


Mathematics  and  Science  Samples,  1988  Assessment 


Mode  of 

Subject      Booklet     Admini-  Time  of     Age      Modal  Sample 

Sample  Code         Sample  Type      Areas       Numbers      stration  Cohort  Testing    Pefn,    Grade  Size 


Mathematics  and  Science 


86:9a 

1986 

bridgp 

RMS 

91-93 

Tape 

Age  9 

Winter 

CY 

4 

6,932 

9[Br86-RMS] 

1988 

bridge 

RMS 

9i-93 

Tape 

Age  9 

Winter 

CY 

4 

3,711 

86:13a 

1986 

bridge 

RMS 

91-93 

Tape 

Age  13 

Fall 

CY 

8 

6,200 

13[Br86-RMS] 

1988 

bridge 

RMS 

91-93 

Tape 

Age  13 

Fall 

CY 

8 

3,942 

MathemaCics 


86:17 

86:17b 

17[Br86-RMS] 


1986  main         RMS  14,36,68    Print         Grade  11/age  17      Spring      Not  CY      11  6,151* 

1986  bridge      RMS  94-95         Tape  Age  17  Spring      Not  CY      11  3,868 

1988  bridge      RMS  61,62,65    Print         Grade  11/age  17      Spring      Not  CY      11  1,852* 


Science 


86:17  1986  main         RMS  47,62,68    Print         Grade  11/age  17      Spring      Not  CY      11  5,611* 

86:17b  1986  bridge      RMS  94-95         Tape  Age  17  Spring      Not  CY      11  3,868 

17[Br86-RMS]        1986  bridge      RMS  63,64,65    Print         Grade  11/age  17      Spring      Not  CY      11  1,862* 


*  Ntiraber  of  age-only  students  who  answered  any  one  of  the  trend  blocks. 


ERIC 


Note:  1)  For  all  three  ages,  mathematics  1988  trend  blocks  are  identical  to  those  administered  in 
1986;  2)  Only  the  subset  of  the  86:17  and  17[Br86-RMS]  samples  that  were  age-eligible  and  received  trend 
blocks  were  used,  and  numbers  on  the  table  reflect  such  samples. 
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Table  15-2 


Mathematics  Weighted  Mean  Proportion  Correct 


Age  17 
(paper) 


Age  17 
(taped) 


Block 

1 

2 
3 

Total 
Noncalculator 

1 

2 
3 

Total 
Noncalculator 


1986  (m 


59.1 
63.4 
65.3 
62.3 
61.0 


60.3 
62.1 
64.5 
62.0 
60.8 


(2211)  • 
(2233)  • 
(2263)  • 
(6157 >  " 


(1934) 
(1934) 
(1934) 
(3868) 


1988  (N) 

61.3  (  619) 
65.7  (  624) 
67,0  (  609) 

64.4  (1852) 
62.7 


No.  of 

Items" 

35 
35 

24  (19) 

94 

75 

35 
35 

24  (19) 

94 

75 


Age  13 
(taped) 


1 

63 

9 

(2075) 

65 

3 

(1405) 

37 

2 

58 

5 

(2054) 

60 

5 

(1281) 

37 

3 

57 

4 

(2071) 

60 

0 

(1256) 

24 

Total 

60 

.3 

(6200) 

62 

.2 

(3942) 

98 

Noncalculator 

61 

.4 

63 

.2 

82 

Age  9 
(tai  -d) 


1 

55 

2 

(2315) 

58 

2 

(1274) 

26 

2 

57 

3 

(2361) 

62 

4 

(1240) 

26 

3 

73 

0 

(2256) 

76 

7 

(1197) 

16 

Total 

60 

2 

(6932) 

64 

2 

(3711) 

68 

Noncalculator 

57 

1 

62 

1 

57 

*    Age-only  BIB  sample  with  at  least  one  mathematics  trend  block. 

^    1986  age  17  trend  sample  blocks  1  and  2  were  paired. 

^    Includes  some  items  that  were  excluded  from  IRT  scaling;  parenthes 
in  this  column  indicate  the  number  of  calculator  items  excluded  from  IRT 
scaling. 


333 


the  1988  assessiment  for  ages  9  and  13  •^fter  the  fic  of  the  1988  data  (  the 
item  parameter.*;  was  examined  visually,  as  well  as  by  me&ns  of  the  ohi-square 
test,  for  every  item.    Consequently,  the  sane  linear  function  as  in  1986  was 
used  to  transforn  provisional  imputed  values  to  the  mathematics  proficiency 
scale.    Although  the  use  of  previously  estimated  item  parameters  was  justified 
at  ages  9  and  13,  this  was  not  the  case  at  age  17.     Hence,  '  x  item  par&meter.s 
applicable  to  age  9  and  age  13  were  kept  unchanged  for  the  nathematics  trend 
analysis;  they  are  presented  in  Tables  F-8  an  ^  F-9  in  Appendix  F, 

The  coexistence  of  item  parameters  that  fit  in  various  degrees  to  the 
data  from  a  particular  year  comes  from  the  need  to  place  several  samples  from 
different  yecrs  on  a  scale  based  upon  common- item  equating.^    When  common- item 
parameters  are  estimated  on  multiple  data  sets,  the  fit  of  the  estimated  item 
response  functions  to  the  weighted  means  of  expected  proportions  correct, 
given  an  ability  level,  is  maximized      Because  of  this  averaging  over  multiple 
data  sets,  it  is  possible  that  the  estimated  item  parameters  will  fit  very 
well  to  the  combined  data  sets  as  a  whole,  but  lej.s  well     j  each  data  sot 
separately.     For  each  item,  fit  of  the  estimated  item  p^    .meters  was  examined 
also  for  the  some  of  the  major  subpopulations ,  such  as  male,  female,  White, 
Black,  and  Hispanic.    Although  a  few  items  indicated  that  the  estimated  item 
parameters  fit  less  well  to  a  particular  subpopulation,  the  examination  of 
these  items  did  not  reveal  any  explanation  for  th3  misfit. 

For  ages  9  and  13,  the  same  common- item  equating  procedure  that  was 
employed  in  the  1986  trend  analysis  was  used  to  align  the  1988  point  to  the 
trend  up  to  1986.    A  brief  description  of  the  procedure  follows.     From  the 
item  parameters  estimated  in  i986  and  background  variables  of  1988,  the 
proficiency  scores  wer*-  imputed  for  the  1988  bridge  data  for  each  age  using 
the  M-'\^OUP  computer  program  based  on  the  plausible  values  methodology 
(Sheehan,  1985;  see  Mis"  1988b,  for  a  detailed  discussion).    Appendix  C 

gives  the  conditioning  Vw  cables  (Table  C-10)  and  t^e  estimated  conditioning 
effects  (Tables  C-38  and  C-39)  for  ages  9  and  13.     The  same  linef.r  constants 
as  in  1986  were  used  to  transform  provisional  imputed  scores  to  the  final 
proficiency  scores  for  mathematics  trend.     The  transformation  constants  for 
all  three  ages  are  listed  in  Table  15-3. 

Table  15-3 

Coefficients  of  the  Linear  Transformation  of  the  Trend  bcale 
from  Original  Units  to  the  Mathematics  Proficiency  Scale 

Ag^e  Intercept  Slope 

9  218.42  35.84 

13  26(..58  34.57 

17  303.25  3] .84 


^The  IRT  equating  of  two  assessments  c^n  be  based  either  on  an  assumption 
that  the  proficiency  distributions  for  the  twc  assessments  are  the  same 
(common-population  equating)  or  an  assumption  that  a  set  of  curjnon  ite.^s  arc 
functioning  in  the  same  manner  in  both  assessments  (common  item  equating). 
The  1988  mathematics  and  science  trend  assessments  used  common- item  equating 
methods.     In  the  future,  we  may  consider  an  alternate  equating  method  based  on 
the  distribu  *.ons  of  item  parameters. 
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Mathematics  Trend  Scaling  for  Age  17 


To  scale  the  age  17  mathematics  trend  data,  new  item  parameters  were 
estimated  using  the  subsample  from  the  1986  BIB  assessment  equivalent  to  the 
1988  trend  sample.    Use  of  the  estimated  item  parameters  in  1986  mathematics 
trend  is  not  appropriate  for  the  1988  assessment  for  age  17,  because  of  the 
different  mode  of  administration  for  the  ?986  and  the  1988  trend  assessments 
for  that  age.    For  example,  on  all  five  items  of  a  type  referred  to  as 
"estimate"  items,  use  of  paper  and  pencil  instead  of  a  tape  recorder  had  a 
dramatic  effect.     "Estimate"  items  ask  the  student  to  select  a.i  answer  among 
several  options,  all  of  which  are  rounded  so  that  none  of  them  is  exactly 
correct.    The  property  of  the  response  options  is  indicated  by  the  word 
"about"  being  positioned  before  "how  much"  or  "how  many"  in  a  question.  When 
an  "estimate"  item  was  presented  under  taped  administration,  enough  time  was 
allowed  for  rough  estimation  of  the  (typically)  large  number,  but  not  enough 
time  was  allowed  for  the  numerical  calculation  of  the  answer.  However, 
because  under  paper-and-pencil  administration  it  is  possible  to  spend  more 
time  to  answer,  the  examinee  may  opt  to  perform  the  calculation  rather  than 
the  estimation.     In  such  a  case,  it  is  more  appropriate  to  treat  an  "estimate" 
item  as  two  different  items  under  different  modes  of  administration.  Figure 
15-1  presents  the  observed  item  regression  curves  of  the  1986  BIB  data  and 
1986  bridge  data  for  one  of  the  "estimate"  items,  along  with  the  item  response 
function  estimated  for  the  1986  bridge  data. 

Therefore,  for  age  17,  both  equating  methods,  common- item  (between  the 
1986  BIB  and  1988  bridge  samples)  and  common-population  (between  the  1986  BIB 
and  1986  bridge  samples),  were  used  to  place  the  1988  trend  sample  on  a  scale 
comparable  to  the  1986  reported  scale.    The  procedure  took  place  as  follows. 
The  item  parameters  for  the  total  set  of  73  items  were  estimated  based  on  the 
two  data  sets:     the  1986  BIB  assessment  and  the  1988  bridge  to  1986.  Both 
samples  included  grade-  and  age -eligible  students  in  order  to  maintain  an 
adequate  sample  size  for  the  estimation  accuracy.    This  resulted  in  a  second 
set  of  item  parameters  for  age  17.     The  new  item  parameters  are  listed  in 
Table  F-10,  Appendix  F;  the  old  parameters  appear  in  Beaton  (1988).  The 
rationale  for  estimating  parameters  for  all  items  instead  of  only  "estimate" 
items  comes  from  the  main  objective  of  the  1988  bridge  to  1986,  namely  to 
examine  the  possibility  of  effects  due  to  changes  '.n  assessment  procedures. 
For  each  item,  fit  of  the  estimated  item  parameters  was  examined  for  some  of 
the  major  subpopulations,  such  as  male,  female,  Wliitc ,  Bi-ck,  and  Hispanic. 

From  th^  above  estimated  item  parameters  and  background  ir*formation  for 
the  appropriate  sample,  proficiency  seres  were  imputed  for  each  student  in 
the  1986  BIB  and  1988  bridge- to -1986  samples.    Appendix  C  gives  the 
conditioning  variables  (Table  C-10)  and  the  estimated  conditioning  effects 
(Table  C-^0)  for  age  17.    Then  the  mean  and  standard  deviation  of  the  imputed 
St  ores  of  the  age-only  subsample  of  the  1986  BIB  were  calculated.  Constants 
were  calculated  and  used  to  match  the  means  and  standard  deviations  of  the 
proficiency  scores  of  the  1986  trend  sample  and  the  age -only  subsample  jf  the 
1986  BIB  sample.     Subsequently,  by  applying  the  same  linear  transformation  to 
the  provisional  imputed  values  of  the  1988  trend  age -only  sample,  the  1988 
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3.^7 


Figure  15-1 

A  Plot  of  Observed  Proportion  Correct  of 
the  1986  BIB  Spiral  and  Trend  A*?sessiaents  with  the  Estimated 
Itea  Response  Function  for  an  "Estimate"  Item 
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trend  point  was  aligned  witn  the  trend  line  up  to  1986.  The  transformation 
constants  for  age  17  data  are  listed  in  Table  15-3. 


15.3    SCALING  OF  THE  SCIENCE  TREND  DATA 

The  1988  science  trend  analysis  followed  procedures  and  methods  similar 
to  those  for  the  mathematics  analysis.     From  the  item  analysis,  it  was  found 
that  the  1988  response  distributions  of  all  response  choices,  including 
"omits,"  were  quite  similar  to  the  1986  data.    The  mean  weighted  proportion 
correct  at  a  block  level  was  computed;  these  values  were  compared  with  the 
1986  results,  and  are  presented  in  Table  15-A. 

In  1986,  item  parameters  wr  :e  estimated  for  the  age  9,  13,  and  17 
samples.     The  trend  items  for  age  13  and  age  17  were  estimated  together 
because  the  majority  of  the  items  were  common  to  both  ages.     For  the  1988 
data,  because  of  the  change  in  the  mode  of  administration  for  age  17,  those 
items  had  to  be  estimated  separately  from  the  age  13  items.    To  obtain  the 
best  estimates  of  proficiencies  for  the  two  years,  items  for  age  13  were 
reestlmated  using  BILOG  (Mislevy  &  Bock,  1982)  on  the  1986  and  1988  bridge 
data  sets.    It  was  found  that,  for  one  of  63  items  in  1986,  "I  don't  know" 
responses  were  treated  as  wrong  when  they  should  have  been  treated  as  "omit." 
This  error  was  found  only  in  the  1986  bridge  data  set  for  age  9.    There  are 
two  reasons  for  judging  that  the  effect  of  this  error  or  the  proficiency  score 
is  very  small:     It  involved  only  8  percent  of  the  responses  for  a  single  item, 
and  the  subjects  who  selected  the  "I  don't  know"  option  had  the  lowest  mean 
proportion  correct  among  all  options.    In  fact,  using  the  trend  item 
parameters  from  1986  estimated  on  the  incorrect  data  sets,  we  compared  the 
means  of  the  ability  distributions  nf  two  data  sets  with  and  without 
correction  of  the  1986  age  9  trend  and  found  that  they  differed  by  about  .07 
on  the  proficiency  scale.    In  order  to  assess  administration  effects  as 
accurately  as  possible,  however,  the  i^em  parfimeters  for  all  items  wrre 
estimated  for  age  9  based  on  the  1986  and  1988  corrected  bridge  data  sets. 
For  each  item,  fit  of  the  estimate'i  item  parameters  was  examined  for  some  of 
the  major  subpopulations,  such  as  male,  female.  White,  Black,  and  Hispanic. 
The  estimated  item  parametars  for  three  ages  are  listed  in  Tables  F-11,  F-12, 
and  F-13  in  Appendix  F.    Although  a  few  items  indicated  that  the  estimated 
item  parameters  fit  less  well  to  a  particular  subpopulation,  the  examination 
of  these  items  did  not  provide  any  clear  explanations. 

The  inrputed  proficiency  values  of  the  1988  sample  were  calculated  from 
the  responses  on  cognitive  items  and  background  questions  based  on  the  item 
parameters  estimated  on  t\e  trend  samples  of  1986  and  1988.    The  imputed 
values  of  the  1988  sample  wore  transformed  to  become  comparable  to  the  trend 
scale  of  198t  .    Note  that  the  1986  sample  was  used  twice  to  obtain  two 
separate  sets  of  trend  item  parameters,  the  one  for  the  data  up  to  and 
;ncluding  1986  and  -he  other  for  the  data  from  1986  a.id  1988.     This  design 
enabled  us  to  use  common-population  equating  baccd  on  the  same  sanple,  anu 
also  to  express  the  difference  in  the  distribution  of  proficiency  between  1986 
and  1988  in  terms  of  the  trend  scale  established  in  1986.    The  lineal 
transformations  were  aerived  separately  for  ages  9  and  13  to  match,  within 
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Table  15-4 


Scierue  Weighted  Mean  Proportion  Correct 

No.  of 


Blof  k 

1986 

(N) 

1988 

(N) 

Iter 

Age  17 

1 

AO 

<; 
J 

(2223)  " 

60.6 

(  634) 

27 

(paper) 

2 

SQ 

fl 
u 

(1935)  • 

60.7 

(  619) 

32 

3 

■J  J 

7 

('2282)  • 

56.3 

(  609) 

23 

Total 

58 

0 

(:6ii)  • 

59.5 

(1862) 

82 

Age  17 

1 

63 

3 

(1934) 

27 

(taped) 

2 

A 
*+ 

(1934) 

32 

3 

O 

(19'4) 

23 

Total 

(3868) 

82 

Age  13 

1 

52 

5 

(2075) 

53,8 

(1405) 

25 

(taped) 

2 

54 

2 

(2054) 

54.7 

(1281) 

31 

56 

2 

(2071) 

57.8 

(1256) 

27 

Total 

54 

3 

(6200) 

55.5 

(3942) 

83 

Age  9 

1 

59 

4 

(2315) 

62.6 

(1274) 

18 

(taped) 

2 

52 

5 

(2361) 

53.5 

(1240) 

25 

3 

68 

5 

(2256) 

69.0 

(1197) 

20 

Total 

59 

5 

(6932) 

61.0 

(3711) 

63 

^    Age -only  BIB  sample  with  at  least  one  science  trend  block. 

^    1986  age  17  trend  sample  blocks  1  and  2  were  paired. 

°    Includes  some  items  that  were  excluded  from  IRT  scaling. 
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each  age  cohort,  the  two  means  and  standard  deviations  of  proficiencies  of  the 
1986  bridge  sample,  one  based  on  the  item  parameters  estimated  on  the  data 
until  1986  and  the  other  based  on  the  item  parameters  estimated  on  the  1986 
and  1988  data.    The  linear  constants  derived  from  those  transformations  were 
applied  to  the  1988  data  set  to  obtain  trend  points  for  1988.    For  age  17,  we 
applied  an  equating  method  identical  to  that  used  for  the  age  17  mathematics 
data.    Appendix  C  gives  the  conditioning  variables  (Table  C-11)  and  the 
estimated  conditioning  effects  (Tables  C-41,  C-42,  and  C-43)  for  all  three 
ages.    Table  15-5  presents  the  linear  coefficients  used  for  the  three  ages. 


Table  15-5 

Coefficients  of  the  Linear  Transformation  of  the  Trend  Scale 
from  Original  Units  to  the  Science  Proficiency  Scale 


Age 


Intercept 


Slope 


9 
13 
17 


225.59 
254.19 
289.34 


41.15 
36.92 
43.05 
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Statistical  Summary  of  1988  NAEP  Data 


341 


lERJC  362 


Chapter  16 


STATISTICAL  SUMMARY  OF  THE  1988  NAEP  SAMPLES  AI3D 
ESTIMATES  OF  THE  PROFICIENCIES  OF  AMERICAN  STUDENTS^ 


Bruce  A.  Kaplan 
Educational  Testing  Service 


The  analysis  of  the  1988  NAEP  data  has  resulted  in  the  production  of 
many  thousands  of  tables  containing  estimates  of  the  proficiency  of  students, 
and  various  subgroups  of  students,  in  American  schools.     This  chapter  gives 
some  selected  results  from  the  assessment  as  well  as  a  statistical  summary  of 
the  1988  NAEP  sample..    The  chapter  assumes  a  general  familiarity  with  the 
structure  of  NAEP  as  summarized  in  the  Introduction  and  the  overview  Chapters 
1  and  7. 

Three  of  the  many  types  of  NAEP  results  are  presented  here: 

•  results  of  the  instrument  development  process,  including  the  sizes 
of  the  item  pools  and  numbers  of  booklets; 

•  results  of  the  sampling  process,  including  the  numbers  of  students 
in  each  sample  by  selected  subgroups;  and 

0         results  of  the  parameter  estimation  process,  including  estimates 
of  the  proficiencies  of  several  populations  of  students  in 
reading,  writing,  civics,  U.S.  history,  geography,  mathematics, 
and  science. 

Interpretive  results  from  the  estimates  presented  here  have  been 
reported  in  the  NAEP  subject  area  trend  and  cross-sectional  reports.    The  1988 
public-use  data  tapes  and  user  guide  (Rogers,  Kline,  Johnson,  Mislevy,  &  ?.ust, 
r 990)  are  available  for  those  who  wish  to  estimate  other  parameters  of  student 
performance  from  the  NAEP  data  or  to  search  for  possible  explanations  for  the 
population  characteristics  that  are  reported  here. 

The  technical  details  ^f  the  estimation  process  that  underlie  these 
tables  are  covered  in  previous  parts  of  this  report  and  not  repeated  here.  A 
detailed  discuss^    *  of  how  to  read  and  use  the  tables  of  background  and 
proficiency  results  is  given  by  Zwick  (1987b). 


^Tho  author  is  indebted  to  Albert  Beaton  for  portions  of  this  chapter. 
Information  for  various  tables  in  this  chapter  was  provided  by  John  Ferris, 
David  Freund,  Lynn  Jenkins,  Edward  Kulick,  Michael  Narcowich,  Norma  Norris, 
Kate  Pashley,  and  Keith  Rust. 
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16.1    IIEASUREMENT  INSTRUMENTS 


In  1988  a  total  of  34  assessment  booklets  and  questionnaires  was  printed 
for  age  class  9,  49  for  age  class  13,  and  53  for  age  class  17.    These  booklets 
are  enumerated  by  age  level  and  by  type  of  meacurement  instrument  in  Tab]e 
16*1 .     Some  of  the  instruments  were  used  at  more  th''*i;  one  age/grade  level. 

The  item  pool  used  to  develop  these  booklets  is  described  in  Table  16  2. 
In  general,  there  are  two  types  of  items,  cognitive  and  rioncognitive .  The 
cognitive  items  are  developed  to  measure  proficiency  In  particular  subject 
areas,  such  as  reading  and  mathematics.     Cognitive  items  may  be  open-ended  or 
multiple-choice.    The  noncognitive  items  are  usually  questions  about  the 
student's  or  teacher's  backgrounds  ^nd  attitudes  but  may  also  ^   obe  other 
areas  such  as  school  policies  or  teaching  methods.    Because  many  items  weie 
used  at  more  than  one  age  class,   ^he  total  number  of  items  in  an  item  pool  is 
not  the  sum  of  the  item  pools  us^d  for  the  three  age  classes. 

All  of  the  ite  s  in  the  subject  area  pools  were  used  for  the  main  NAEP 
assessment,  but  not  ail  could  be  used  for  the  various  bridge  assessments. 
Table  16-3  shows  the  number  of  cognitive  items  in  each  subject  area  that  were 
used  in  the  separate  samples. 

The  excluded  student,  teacher,  and  school  questionnaires  contained  only 
noncognitive  questions.    The  number  of  items  in  the  noncognitive  pools  is  the 
same  as  the  number  of  items  in  the  questionnaires.    More  information  about  the 
instruments  that  were  developed  is  provided  in  Chapters  2  and  4. 

16.2     SAMPLE  CHARACTERISTICS 

In  this  section,  the  characteristics  of  the  final  NAEP  sample  is 
described.    The  process  by  which  the  sample  was  selected  is  discussed  in 
Chapter  3. 

In  the  1988  main  assessment,  NAEP  contacted  1,29    schools,  of  which 
1,030  contributed  data  to  the  assessment.    The  disposition  of  these  schools  is 
shown  in  Table  16-4.     Some  of  the  schools  were  unwilling  to  cooperate;  others 
were  believed  to  be  eligible  from  the  sampling  frame,  but  were  not.  The 
cooperation  rate  is  calculated  as  the  sum  of  cooperrting  schools  and  the 
schools  that  were  found  to  have  no  eligible  students  divided  by  the  same  sum 
plus  the  schools  that  refused  or  were  from  districts  that  refused  to 
cooperate . 

Table  16-4  also  shows  the  number  of  schools  in  several  categories: 
region  of  the  country  (northeast,  southeast,  central,  west),  school  governance 
(public,  private.  Catholic,  Bureau  of  Indian  Affairs,  Department  of  Defense), 
size  and  type  of  community,  degree  of  urbanicity,  grade  span  of  school,  number 
f  teachers,  and  number  of  students. 

For  the  1988  bridge  (trend)  studies,  NAEP  contacted  568  schools,  of 
which  441  contributed  data  to  the  v      jus  bridge  assessments.    Table  16-5 
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supplies  the  same  information  for  the  schools  assessed  for  the  bridge  ^dies 
that  the  previous  table  supplies  for  the  main  assessment  schools. 

The  numbers  of  respondents  to  the  teacher  questionnaire  are  summarized 
in  Table  16  6.    The  first  column  in  this  table  includes  the  number  of  teachers 
who  responded  by  ege  class  and  subject  area.    The  final  column  contains  the 
number  of  students  who^e  teachers  responded  to  the  questionnaire. 

NAEP  is  administered  in  units  called  assessment  sessions,     if.  the  number 
of  students  attending  an  assessment  session  is  less  than  a  predetermined 
number,  the  students  missing  from  the  session  are  assigned  to  a  makeup  se.*-  ion 
and  then  assessed.    Table  16-7  shows  the  number  of  regular  and  makeup  sessions 
in  1988  MEP  by  a^e  diss  for  the  main  NAEP  and  two  bridge  samples. 

Altogether,  133,542  student.s  were  involved  in  1988  NAEP,  including 
excluded  students.    The  breakdown  by  a«y.e  class  and  b^   sample  is*  shown  in  Table 
16-8 

Tables  16-9  through  16-11  display  the  distribution  of  the  students 
assessed  in  the  mai.i  NAEP  assessment  in  several  basic  categories  for  the  three 
age  classes:     gender,  racial/athnic  grouping,  region  of  the  country,  parental 
education,  and  size  and  type  of  community.    These  tables  have  four  columns: 

t         eligible  by  age,  which  means  that  the  students  were  in  an 
appropriate  age  group; 

•  eligible  by  grade,  which  means  that  the  students  were  in  an 
appropriate  grade; 

•  eligible  by  age  and  by  grade,  which  means  that  the  students  were 
of  both  an  appropriate  age  and  appropriate  grade;  and 

•  eligible  by  age  or  by  jjrade,  which  is  the  total  number  of  students 
for  whom  data  were  collected. 


Tables  16-12  through  16-19  contain  the  distribution  of  students  in  f  a 
same  categories  by  age  class  for  the  bridge  samples.    Tables  16-12  to  16  1 
contain    he  distributions  for  the  bridge  to  1984  sample.    Tables  16-15  to 
16-17  dic^play  the  distributions  for  the  bridge  to  1986  sample.     Table  16-18 
and  Table  16-19  entimerate  the  students  in  the  two  age  classes  assessed  as  part 
of  the  civics  bridge. 

Similarly,  Tables  16-20  through  16-23  contain  the  disuribution  of 
excluded  students  by  age  class.    The  distribution  for  the  excluded  students  in 
the  main  and  bridge  samples  combined  are  displayed  in  Tables  16-20  and  16-21 
for  grade  4/age  9  and  grade  8/age  13,  respectively.    Table  16-22  contains  the 
distribution  of  excluded  students  for  the  main  sample  grade  12/age  17 
students,  while  Table  16-23  contains  the  distribution  of  excluded  students  for 
the  bridge  samples  of  grade  11/age  17.    These  two  samples  could  not  be 
combined  because  of  different  age  definitions. 


3.^5 
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16.3    POPULATION  ESTIMATES 


The  1988  NAEP  samples  were  designed  fcr  e'^timatinp  the  size  and 
attributes  of  a  number  of  different  populations  >f  students.    The  estimation 
procedures  use  sampling  weights,  developed  by  Westat,  Inc  ,  that  are  used  in 
conjunction  with  the  members  of  the  sample  (see  Chapter  iJ)  .     In  this  chapter, 
all  estimates  of  population  parameters  use  these  sampling  weights. 

Table  16-24  shows  the  sizes  of  the  various  samples  and  the  sums  of  their 
sampling  weights  by  •^rade/age.     In  most  cases,  the  sum  of  the  weights  for  a 
given  sample  is  an  es>cimate  of  the  number  of  students  who  are  in  the 
population  represented  by  ,.he  sample.     In  other  words,  the  sum  of  the  weights 
is  an  estimate  of  the  number  of  eligible  stud      s  in  the  grade  or  age  of 
interest. 

The  main  assessment  can  be  divided  into  winter  and  spring  subsamples. 
These  subsamples  each  have  their  own  sets  of  ^eight^,  as  well  as  a  set  of 
weights  that  allows  the  combination  of  the  two  subsamples.    The  sample  sizes 
and  the  estimated  population  sizes  for  the  winter  and  spring  subsamples,  as 
well  as  the  combined  main  assessment  sample  are  provided  in  Table  16-24.  The 
combined  set  of  weights  will  be  used  in  subsequent  tables. 

Note  that  the  samples  for  the  main  asse    .*flent,  the  samples  for  all  three 
age  classes  of  the  bridge  to  1984,  and  the  samples  for  the  oldest  age  class 
for  the  bridge  to  1986  are  grade  and  age  samples.    The  samples  for  the  younger 
K      age  classes  of  the  bridge  to  1986  and  the  samples  for  the  civics  bridge 
are  age- only  samples. 

The  sum  of  the  weights  of  the  excluded  students  estimates  the  number  of 
ineligible  students  in  the  respective  grade/age  class.    The  17-year-old 
excuded  students  are  split  into  two  samples,  one  for  which  the  modal  grade  is 
12  (for  the  main  samples),  and  one  for  which  the  modal  grade  is  11  (for  the 
bridge  samples).    Thi^  split  was  necessary  because  of  differences  in  the  time 
of  testing  and  age  definitions  use  a  for  the  bridge  and  main  samples  (see 
Chapter  3) . 

In  most  cases,  the  number  of  students  in  a  grade/age  combination  is  not 
of  interest;  a  researcher  will  be  interested  in  estimating  the  number  c€ 
students  at  either  a  grade  or  an  age  level.      For  the  samples  that  contain 
both  grade-  and  age-eligible  students,  an  estimate  of  the  number  of  students 
at  an  age  level  can  be  made  by  summing  the  weights  of  only  the  age -eligible 
students  and  adding  the  corresponding  sample  of  age -eligible  exc'  ^ded 
students.    An  estimate  of  the  number  of  students  in  a  grade  samp  a  can  be  made 
by  summing  the  weights  of  grade-eligible  student.,  plus  the  weights  of  grade- 
eligible  students  from  the  appropriate  excluded  student  sample. 

From  the  main  NAEP  sampler,  the  next  group  of  tables  estimates  how  many 
students  are  age-eligible  and  grade-eligible  by  age  class.     Tables  16-25 
through  16-27  show  how  many  st  dents  at  a  ptrticular  grade  level  arc  at,  in, 
or  above  the  modal  age  for  thc*L  grade,  and  how  many  at  j  particular  age  level 
are  at,  in,  or  above  the  modal  grade  for  that  age.    Alonj;  with  the  counts  from 
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these  samples,  the  sum  of  the  weights  (Weighted  N)  for  each  category  is 
presented,  and  these  suras  are  estimates  of  the  numbers  of  students  in  these 
categories  in  the  population.    The  standard  errors  of  these  estimates  and 
coefficients  of  variation  are  .      »  given.     (The  coefficient  of  variation  is 
defined  as  100  times  the  standai-d  error  divided  by  the  weighted  N.) 

Tables  16-28  through  16-34  contain  the  same  type  of  information  for  the 
several  bridge  booklets,  by  age  level.    Tables  16-28  to  16-30  have  the 
identical  format  as  the  main  assessment  tables,  but  contain  the  information 
for  the  bridge  to  1984.     Table  16-33  is  also  of  the  same  format,  and  contains 
the  data  for  grade  11/age  17  bridge  to  1986.    Table  16-31  and  16-32  are  for 
the  other  two  age-only  samples  of  the  bridge  to  1986.     Since  these  are  age- 
only  samples,  the  partitioning  of  the  sample  by  modal  age  groupings  provides 
no  added  information.     Instead  we  supply  information  by  booklet.     This  is 
useful  because  each  booklet  was  weighted  so  that  the  sura  of  the  weights  of  all 
students  completing  a  booklet  estimates  the  population  size.    Table  16-34  is 
constructed  for  the  civics  bridges.    These  are  also  age -only  samples,  with  one 
booklet  per  ago  group,  so  the  information  can  be  provided  on  one  table. 

The  next  tables  show  the  sizes  of  the  estimated  populations  of 
assessable  students  and  the  weighted  percentages  for  various  NAEP  reporting 
categories,    lliese  categories  include  gender,  racial/ethnic  grouping,  region 
of  the  country,  parents'  education,  and  size  and  type  of  community.  The 
estimated  subpopulation  percentages  for  the  main  NAEP  samples  are  shown  in 
Tables  16-35  through  16-37,  separately  by  age  eligibility,  grade  eligibility, 
and  grade/age  eligibility.    Tables  16-38  to  16-45  show  the  same  information 
for  the  bridge  samples.     In  a  similar  manner,  Tables  16-46  to  16-49  show  the 
estimated  total  population  of  excluded  students  and  the  weighted  percentages 
by  demographic  subgroups. 

Students  were  assigned  proficiency  values  in  a  subject  area  only  if  they 
received  at  least  one  assessment  block  in  that  area,  and  thus  the  sample  sizes 
of  students  who  have  proficiency  values  vary  from  one  subject  area  to  another. 
Tables  16-50  through  Table  16-60  show  the  number  of  students  with  proficiency 
values  in  er^ch  subject  area  by  age  and  grade  combinations. 

Tables  16-61  to  16-70  contain  population  estimates  of  student 
proficiencies  by  grade  and  by  the  subpopulations  of  gender  and  race/e thnii.  it> 
The  information  about  proficiency  includes  the  mean  and  standard  deviation  of 
each  subpopulation  as  well  as  the  value  of  the  5th,  10th,  2bth,  50th  (median), 
75th,  90th,  and  95th  percentiles.     Results  are  shown  separately  for  each 
subject  area.     Standard  errors  of  the  estimates  ire  included  in  parentheses. 

Tables  16-71  through  16-109  contain  results  for  more  finely  defined 
subpopulations.    The  major  reporting  variables  (gender,  race/ethnicity, 
parental  education)  are  cross-classified  with  one  another.     For  example,  Table 
16-7?  cross-classifies  gender,  racial/ethnic  grouping,  and  parental  education 
with  the  racial/ethnic  grouping  for  fourth  graders.     Information  included 
about  these  subpopulations  is  the  actual  sample  size,  the  estimated  population 
size  (and  its  coefficient  of  variation),  the  proportion  of  students  in  each 
subpopulation  (and  its  standard  error),  and  the  average  prof ienc>    >f  the 
students  (and  its  standard  error) . 

347 


Table  16-1 


Msasurement  Instruments  Developed  for  1988  NAEP 


Ins truments 

Student  Assessment  Booklets 

Main  Sample 
Bridge  to  1984 
Bridge  to  1986 
Civics  Bridge 

Total 


Age  Class 
9         13  17 


22 

36 

37 

6 

6 

6 

3 

3 

7 

0 

1 

1 

31 

46 

51 

Questionnaires 

Excluded  Student  Questionnaire  111 

Teacher  Questionnaire  110 

School  Characteristics  Questionnaire  111 

Total  3         3  2 
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Table  16-2 


Numbers  of  Distinct  Itema  Administered,  by  Age  Class 

  AGE  CLASS   


Conunon  Baukgiound 

Reading 

Backgro  \d  and  Attitude 
Cogniti- 

Writing 

Background  and  Attitude 
Cognitive 

Civics 

Background  and  Attituu^ 
Cognitive 

U.S.  History 

Background  and  Attitude 
Cognitive 

Geography 

Background  and  Attitude 
Cognitive 

Document  Lito--acy 

Background  and  Attitude 
Co^initive 

MathematiJs 

Background  and  Attitude 
Cognitive 

Science 

Background  and  Attitude 
Cognitive 

Excluded  Studeat  Questionnaire 

Teacher  Questionnaire 

School  Characteristics 

and  Policies  Questionnaire 

Total  Distinct  Items 


i 
66 

125 
168 

45 
16 

9 
51 

9 
45 

0 
0 

0 
0 

3 
68 

16 
63 

67 

C7 

159 
989 


11 
68 

137 
187 

80 
17 

25 

168 

27 
161 

0 
0 

0 
57 

29 
84 

28 
5d 

67 

91 

164 
1432 


U 
98 

193 
210 

86 
17 

40 
174 

62 
211 

28 
78 

0 

57 

69 
142 

45 
110 

67 

0 

171 
1844 


Total 
Distinct 
I  terns* 

110 

229 
384 

89 
35 

74 
239 

98 

290 

28 

78 

0 
57 

79 

222 

61 
195 

67 

131 

234 
2674 


*  Because  many  items  were  used  at  more  than  one  age  class,  the  total 
number  of  distinct  items  is  not  ♦"he  sum  of  distinct  items  used  for  the  three 
age  classes. 


•-49 


3«n 


ERIC 


Table  16-3 


Numbers  of  Distinct  Cognitive  Items 
by  Age  Class  and  Sample  Typo 


Reading 

Cross- sectional 
Lridge  to  1984 
Bridge  to  1986 

Writing 

Cross- sectional 
Bridge  to  1984 

Civics 

Cross- sectional 
Civics  Bridge 

U.S.  History 
Cross- sect  lal 
Bridge  to  .    J 6 

Geography 

Cross- sectional 

Document  Literacy 
Cross- sectional 

Mathematics 
Bridge  to  1986 

Science 

Bridge  to  1986 


83 
105 
31 


10 
6 


51 
0 


45 
0 


{^GE  CLASS 


100 
108 
35 


11 
6 


155 
96 


161 

0 


68 
63 


57 


84 


58 


12 


110 
96 
72 


11 
6 


152 
101 


162 
105 


78 
57 
142 
110 


Total 

OistiniJt 
Items* 


215 
193 
91 


23 
12 


221 
127 


241 
105 


78 


57 


222 


195 


*  Because  many  items  were  used  ;t  more  than  one  age  class,  the  t-tal 
number  of  dist5.nct  items  is  not  the  sum  of  distinct  items  jsed  for  the  thr*  > 
age  classes. 


ERIC 


350 

370 


Table  16-4 


Characteristics  of  Schools 

in  Main  NAEP 

(Cross- 

•Sf  ctional) 

Samples 

  Grade/Age   

8/13 

12/17 

i<  tai 

520 

394 

COOP£RATING 

327 

399 

304 

1030 

OUT-OF  RANGE  OR  CLOSED 

11 

14 

11 

36 

NO  ELIGIBLES  ENROLLED 

4 

39 

13 

56 

ULo  IKlUi    KCiT  UoCiU 

40 

46 

110 

SCHOOL  REFUSED 

9 

28 

?0 

57 

COOPLRATION  RATE 

88.7 

86.6 

82.8 

86. 

REPLACEMENTS 

REPLACEMENTS  FOR  REFUSALS 

15 

18 

23 

56 

REPLACEMENTS  COOPERATING 

9 

14 

8 

31 

TOTALS 

UUUrCiKAi  La\j  buHUULb 

413 

312 

lUol 

COMPLETING  QUESTIONNAIRES 

316 

O  O  A 

389 

O  O  A 

289 

994 

REGION 

NORTHEAST 

69 

87 

60 

216 

SOUTHEAST 

8f 

105 

o  c 
o5 

276 

CEl  iRAL 

81 

lOz 

79 

262 

YiJLo  1 

iiy 

o  o 
00 

SCHOOL  TYPE 

PUBLIC 

282 

JZi 

III 

875 

PRIVATE 

13 

33 

19 

65 

CATHOLIC 

41 

59 

21 

121 

BUREAU  OF  INDIAN  AFFAIRS 

0 

0 

0 

0 

DEPT  OF  DEFENSE 

0 

0 

0 

0 

SIZE  AND  TYPE  OF  COMMUNITY 

RURAL 

34 

42 

30 

106 

LOW  METRO 

37 

45 

32 

114 

HIGH  METRO 

35 

41 

31 

107 

BIG  CITY 

47 

56 

44 

147 

FRINGE 

43 

68 

35 

146 

MEDIUM  CITY 

49 

48 

29 

126 

SMALL  PLACE 

91 

113 

111 

315 

351 

371 


Table  16-4  (continued) 


Characteristics  of  Schools  in  Main  NAEP  (Cross- sectional)  Jamples 

  Grade/Age   


URBANICITY 
URBAN 
SUBURBAN 
RURAL 

GRADE  SPAN 

KINDERGARTEN  TO  GRADE  12 
KINDERGARTEN  TO  GRADE  6 
KIIIDERGARTEN  TO  GRADE  8 
GRADE  6  OR  7  TO  GRADE  8 
GRADE  7  TO  GRADE  9 
GRADE  7  TO  GRADE  12 
GRADE  9  TO  GRADE  12 
GRADE  10  TO  12 
KINDERGAPTEN  TO  GRADE  3 

NUMBER  OF  TEACHERiJ 
UNKNOWN 


1  - 
5  - 
10  - 


4 
9 
19 


20  -  49 


50  - 
75  - 
100  + 


74 
99 


NUMBER  OF  STUDENTS 
UNKNOWN 


1  - 

100  - 

300  - 

500  - 

750  - 

1000  - 

150C  + 


99 
299 
499 
749 
999 
1499 


4/9 


115 
125 
95 


18 
228 

68 
6 
0 
0 
0 
0 

15 


1 
6 
28 
124 
166 
11 
0 
0 


1 
10 
93 
121 
71 
29 
10 

1 


8/13 


137 
149 
126 


36 
22 
120 
144 
46 
30 
14 
0 
0 


1 
12 
40 
85 
199 
55 
12 
9 


1 
18 
103 
73 
87 
69 
45 
17 


12/17 


87 
106 
118 


50 
0 
0 
0 

3 
26 
187 
45 
0 


1 
3 

11 
37 
103 
54 
49 
54 


1 

9 
52 
42 
38 
26 
61 
83 


Total 


339 
380 
339 


104 
250 
188 
150 
49 
56 
201 
45 
15 


3 
21 
79 
246 
468 
120 
61 
63 


3 

37 
248 
236 
196 
124 
116 
101 


mc 


352 


372 


Table  16-5 


CharaccexiStics  of  Schools  in  NAEP  Bridge  Samples 

  Age  Class   


TOTAL  ORIGINAL  SAMPLE 
COOPERATING 

OUT-OF  RANGE  OR  CLOSED 
NO  ELIGIBLES  ENROLLED 
DISTRICT  REFUSED 
SCHOOL  REFUSED 

COOPERATION  RATE 


190 
154 
10 
3 
12 
11 

87.2 


13 

223 
173 
4 
30 
10 
6 

92.7 


17 

135 
114 
0 
7 
21 
13 

78.1 


Total 

568 
441 
14 
40 
43 
30 

86.8 


REPLACEMENTS 

REPLACEMENTS  FOR  REFUSALS 
REPLACEMENTS  COOPERATING 


7 
2 


12 
8 


11 
6 


30 
16 


TOTALS 

COOPERATING  SCHOOLS  '  j6  181  120  457 

COMPLETING  QUESTIONNAII^IES  152  167  110  429 


REGION 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 


41 
37 
38 
40 


38 
48 
57 
38 


26 
27 
25 
42 


i05 
112 
120 
120 


SCKOC'^  TYPE 
PUBLIC 
PRIVATE 
CATHOLIC 

BUREAU  OF  INDIAN  AFFAIRS 
DEPT  OF  DEFENSE 


1?4 
12 
20 
0 


130 
23 
28 
0 


95 
10 
15 
0 


349 
45 
63 
0 


SIZE  AND  TYPE  OF  COMMUtllTY 
RURAL 
LOW  METRO 
HIGH  HETRO 
BIG  CITY 
FRINGE 
MEDIUM  CITY 
SMALL  PLACE 


13 
16 
24 
15 
22 
16 
50 


19 
17 
24 
11 
25 
21 
64 


8 
2 
22 
22 
18 
15 
33 


40 
35 
70 
48 
65 
52 
147 


353 


Table  16-5  (continued) 


Characteristics  of  Schools  in  NAEP  Bridge  Samples 


—  Age  Class  - 

n  u 


URBANICITY 
URBAN 
SUBDP3AN 

rur:  , 

grade  span 

kindergarten  to  grade  12 
kindergarten  to  grade  6 
kindergarten  to  grade  8 
grade  6  or  7  to  grade  8 
grade  7  to  grade  9 
grade  7  to  grade  12 
grade  9  to  grade  12 

TRADE  10  TO  12 
KINDERGARTEN  TO  GRADE  3 

NUHBER  OF  TEACHERS 
UNC^ \SSIFIED 


1  - 
5  - 
10  - 
20  - 
50  - 
75  - 
100  + 


4 
9 
19 
49 
74 
99 


NUMBER  OF  STUDENTS 
UNCLASSIFIED 
1  -  99 
100  -  299 
:00  -  499 
500  -  749 
750  -  999 
1000  -  1499 
1500  + 


48 
66 
41 


12 
95 
39 
3 
0 
0 
0 
0 
7 


0 
4 
20 
49 
77 
6 
0 
0 


0 
5 
44 
56 
24 
12 
4 
1 


46 
73 
62 


25 
10 
62 
44 
12 
13 
15 
0 
0 


0 
5 
20 
43 
79 
23 
5 
6 


0 
8 
56 
40 
27 
16 
25 
9 


39 
50 
31 


12 
0 
0 
0 
0 
10 
81 
17 
0 


0 
2 
5 
14 
35 
28 
17 
19 


0 
3 
21 
10 
10 
17 
18 
41 


Total 


133 
189 
134 


49 
105 
101 
47 
12 
23 
96 
17 
7 


0 
11 
45 
106 
191 
57 
22 
25 


0 
16 
121 
106 
71 
45 
47 
1,1 


ERIC 


354 


374 


Table  16-6 

Numbers  of  Responses  to  Teacher  Questionnaire 


Sample 
Grade  4  READING 
Grade  8-  WRITING 


Number  of 
Teachers  Responding^ 

769 

756 


Number  of  Students  with 
Responding  Teachers 

3901 

3570 


355 


Table  If -7 

Numbers  of  Assessment  Sessions  by  Sample,  Type  of  Session,  and  Age  Clai  s 


—  Age  Class  - 
13  17 


Total 


MAIN  SAMPLE* 
REGULAR 
MAKEUP 

BRIDGE  TO  1984  SAMPLE 
REGULAR 
MAKEUP 

BRIDGE  TO  1986  SAMPLE 
REGULAR 
MAKEUP 

CIVIlJS  BRIDGE 
REGULAR 
MAKEUP 


582 
0 


184 
1 


335 
1 


0 
0 


635 
15 


185 
1 


217 
0 


105 
0 


511 
68 


110 
23 


131 
24 


97 
20 


1728 
83 


479 
25 


683 
25 


202 
20 


TOTAL 


REGULAR 

MAKEUP 

COMBINED 


1101 
2 

1103 


1142 
16 
1158 


849 
13S 
984 


3092 
153 
3245 


*  Includes  focused-BIB,  intercorrelation,  ard  document  literacy  samples. 


|er!c 


Table  16-8 


Numbers  of  Students  Assessed  and  Excluded 
by  Sample  and  Age  Class 


Age  Class 


13 


17 


Total 


ASSESSED 


MAIN  NAEP 

23012 

31601 

32710 

87323 

BRIDGE  TO  1984 

5188 

5500 

4622 

15310 

BRIDGE  TO  1986 

3711 

3942 

7052 

14705 

CIVICS  BRIDGE 

(TO  1976  AND  1982) 

0 

1938 

1786 

3724 

EXCLUDED 

MAIN  NAEP 

1554 

2213 

1527 

5294 

BRIDGES 

699 

851 

538 

208r 

TOTAL 

34164 

46045 

48235 

128444 

*  Includes  focused-BIB,  intercorrelation,  and  doci"".ent.  literacy  samples, 


357 


Table  16-9 


Ntunbers  of  Students  in  Main  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  A/Age  9 


Eligible  by 


TOTAL 


SEX 


Male 
Female 


RACE/ETIINICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
West 

PARENT'S  EDUCATION 

Lesc  than  High  School 
High  School 

Greater  High  School 

Graduate  lege 
Unknown 

SIZE  AND  TYPE  OF  COMMUNITY 
Rural 

Distidvantaged  Urban 
Advantaged  Urban 
Big  City 
Fringe 
Medium  City 
Small  Places 


Age 
16366 


8227 
8L39 


9335 
2831 
3336 
864 


3099 
4559 
3349 
5359 


797 
2300 
1147 
5759 
6273 


1422 
1917 
1990 
2332 
2316 
2442 
3947 


Grade  Age  &  Grade 
17137  10491 


8708 
8425 


9645 
3104 
3476 
912 


3061 
►897 
.1518 
5661 


942 
2453 
1333 
6156 
6163 


1432 
2042 
2078 
2679 
2385 
2500 
4021 


4966 
5525 


6138 
1823 
1956 
574 


2163 
2732 
2127 
3469 


438 
1411 

831 
4042 
3712 


859 
1178 
1411 
1609 
1593 
1523 
2318 


Age  or  Grade 
23012 


11969 
11043 


12842 
4112 
4856 
1202 


3997 
6724 
4740 
7551 


1301 

3342 
1649 
7883 
8724 


1995 
2781 
2657 
3402 
3108 
3419 
5650 


358 


378 


ERIC 


Table  16-10 


Nvimbers  of  Students  in  Main  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  8/Age  13 


TOTAL 


Age 
?.2471 


SEX 


Male 
Female 


10923 
11548 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
West 

PARENT'S  EDUCATION 

Less  than  High  School 
High  School 

Greater  than  High  School 
Graduated  College 
Unknown 

SIZE  AND  TYPE  OF  C  'MUNITY 
Rural 

Disadvantaged  Urban 
Advantaged  Urban 
Big  City 
Fringe 
Medium  City 
Small  Places 


14008 
3702 
3659 
1102 


4'="4 
6022 
4943 
6922 


1890 
5937 
3951 
8401 
2224 


1331 
2634 
2098 
3179 
4465 
3275 
5489 


Eligible  by 


Grade  Age  &  Grade 
23801  14671 


Age  or  Grade 
31601 


11804 
11997 


6608 
8063 


16119 
15482 


14719 
4153 
3769 
'160 


9643 
2248 
2046 
734 


19084 

5607 
538Z 
1528 


4843 
6484 
5102 
7372 


3319 
3695 
3115 
4542 


6108 
8811 
6930 
9752 


215"^ 
6328 
4356 
878» 
2100 


1015 
3685 
2895 
5932 
1102 


3028 
8580 
5412 
11257 
322? 


1399 
2738 
2325 
3223 
4768 
3408 
5940 


790 
1525 
1580 
1996 
3267 
2088 
3425 


19/4  0 
J847 
2843 
4406 
5966 
4595 
8004 


359 

379 


Table  16-11 


Numbers  of  Students  in  Main  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  12/Age  17 


Eligible  by 


TOTAL 


SEX 


Male 
Female 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
West 

PARENT'S  EDUCATION 

Less  than  High  School 
High  School 

Greater  than  High  School 
Graduated  Colle 
Unknovm 

SIZE  AND  TYPE  OF  COMMUNITY 
Rural 

Disadvantaged  Urban 
Advantaged  Urban 
Big  City 
Fi-inge 
Medium  City 
Small  Places 


Age 
25531 


12361 
13170 


17273 
4645 
2770 
843 


4719 
7857 
5869 
7086 


2398 
6522 
6235 
9581 
704 


1131 
2647 
3152 
4572 
3357 
2705 
7967 


Grade  Age  &  Grade 
24316 


11571 
12745 


16670 
4224 
2614 
808 


4514 
7293 
5831 
6678 


2187 
6014 
5974 
9469 
582 


1090 
2300 
3360 
4380 
3148 
2572 
.'466 


17137 


7527 
9610 


12224 
2808 
1576 
529 


3447 
5170 
3916 
4604 


1280 
4041 
4440 
7004 
322 


675 
1500 
2450 
3203 
2257 
1878 
5174 


Age  or  Grade 
32710 


16405 
16305 


21719 
6061 
3808 
1122 


5786 
9980 
778^ 
9160 


3305 
8495 
7769 
12046 
964 


1546 
3447 
4062 
5749 
4248 
3399 
10259 


360 


3  SO 


Table  16-12 


Numbers  of  Students  in  Bridge  to  1984  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  4/Age  9 


TOTAL 


Age 
3782 


SEX 


Male 
Female 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
West 

PARENT'S  EDUCATION 

Less  than  High  School 
High  School 

Greater  than  High  School 
Graduated  College 
Unknovm 


1863 
1919 


2245 
693 
688 
156 


969 
993 
801 
1019 


159 
561 
189 
1545 
1321 


SIZE  AND  TYPE  OF  COMMUNITY 

Rural  283 

Disadvantage,   Urban  346 

Advantaged  Urban  617 

Big  City  380 

Fringe  576 

Medium  City  424 

Small  Places  1156 


Eligible  by 


Grade     Age  &  Grade 


3979 


2573 


Age  or  Grade 
5188 


1984 
1995 


1167 
1406 


2680 
2508 


2305 
782 
733 
159 


1530 
481 
456 
106 


3020 
994 
965 
209 


1048 
1127 
783 
1021 


763 
642 
479 
689 


1254 
1478 
1105 
1351 


191 
663 
195 

1679 
1242 


102 
382 
130 
1136 
820 


248 
842 
254 
2088 
1743 


304 
396 

610 
412 
592 
444 
1221 


186 
211 
424 
27  i 
433 
289 
757 


401 
531 
803 
519 
735 
579 
1620 


361 


3S1 


Table  16-13 


Number,  of  Students  in  Bridge  to  1984  Sbinple 
by  Type  of  Eligibilirv  and  Subgroup  Classification,  Grade  8/Age  13 


TOTAL 


SEX 


Male 
Female 


EACE/ETHNICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
West 

PARENT'S  EDUCATION 

Less  than  High  School 
High  School 

Greater  than  High  School 
Graduated  College 
Unknown 

SIZE  AND  TYPE  OF  COMMUNITY 
Rural 

Disadvantaged  'irban 
Advantaged  Urban 
Big  City 
Fringe 
Medium  City 
Small  Places 


Age 
4005 


1964 
2041 


2890 
576 
358 
181 


884 
840 
1199 
1082 


299 
1227 

414 
1686 

369 


257 
357 
C18 
2  1 
715 
375 
1422 


  Eligible  by  — 

Grade      Ape  6e  Grade 


413  i 


2024 
2109 


2958 
619 
378 
178 


904 
887 
1206 
1136 


325 
1271 

466 
1724 

333 


240 
371 
655 
284 
715 
395 
1473 


2638 


1160 
1478 


1956 
357 
214 
111 


651 
532 
746 
70S 


168 
785 
109 
1173 
197 


166 
207 
440 
174 
469 
281 
901 


Age  or  Grade 
5500 


2828 
2672 


3892 
838 
522 
248 


1137 
1195 
1659 
1509 


456 
1713 

571 
2237 

505 


331 
521 
823 
381 
961 
489 
1994 


ERIC 


362 

38 


Table  16-14 


Numbers  of  Students  in  Bridge  to  1984  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  11/Age  17 


TOTAL 
SEX 

Male 
Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
West 


Age 
3652 


1667 
1985 


2577 
638 
281 
156 


792 

642 

1274 


PARENT'S  EDUCATION 

Less  than  High  School  29V 

High  School  1051 
Greater  than  High  School  636 

Graduated  College  1564 

Unknown  96 

SIZE  AND  TYPE  OF  COMMUNITY 

Rural  159 

Disadvantaged  Urban  51 

Advantaged  Urban  663 

Big  City  702 

Fringe  632 

Medium  City  483 

Small  Places  962 


Eligible  by 


Grade      Age  &  Grade 
3664  2694 


Age  or  Grade 
4622 


1553 
2011 


1179 
1515 


2141 
2481 


2620 
631 
275 
138 


1995 
424 
177 
98 


320r> 
845 
379 
196 


821 
959 
660 
1224 


584 
682 
504 
924 


1029 
1221 
798 
1574 


284 
1041 

654 
1580 
99 


174 
755 
498 
1211 
53 


407 
1337 

792 
1933 

142 


153 
62 
684 
736 
628 
473 
92t' 


126 
32 
521 
499 
485 
355 
676 


186 
81 
826 
939 
775 
601 
1214 


ERIC 


363 


Table  16-15 


Numbers  of  Students  in  Briuge  to  1986  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  9 


Eligible  by 


TOTAL 
SEX 

Hale 
Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
Vest 

PARENT'S  EDUCATION 

Less  than  High  School 
Hi.gh  School 
Greater  then  High  Schc 
Graduated  College 
Unknown 

SIZE  AND  TYPE  OF  COMMUNITY 
Rura.l 

Disadvantaged  Urban 
Advantaged  Urban 
Big  City 
Fringe 
Medium  City 
Small  Places 


Age 
3711 


1837 
1874 


2225 
586 
726 
17/» 


946 
1043 
801 
921 


141 
545 
281 
1571 
1145 


279 
349 
636 
350 
535 
411 
1151 


Grade      Age  &  Grade 


2498 


1160 
1338 


1560 
397 
431 
110 


759 
639 
507 
593 


87 

378 
209 
1134 
679 


180 
213 
406 
249 
408 
269 
773 


2498 


1160 
1338 


1560 
397 
431 
110 


759 
639 
507 
593 


87 
378 
209 
1134 
679 


180 

406 
249 
408 
269 
77.\ 


Age  or  Grade 
3711 


1837 
1874 


2225 
586 
726 
174 


946 
1043 
801 
921 


545 
281 
1571 
1145 


279 
349 
636 
350 
535 
411 
1151 


364 


ERIC 


3H4 


Table  16-16 


Numbers  of  Students  in  Bridge  to  1986  Sample 
by  Tjrpe  of  Eligibility  and  Subgroup  Classification,  Age  13 


TOTAL 
SEX 

Male 
Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


Age 
3942 


1963 
1979 


2756 
639 
388 
159 


REGION 

Northeast 
Southeast 
Central 
West 


905 
843 
1104 
1090 


PARENT'S  EDUCATION 

Less  than  High  School  278 

High  School  992 

Greater  than  High  School  619 

Graduated  College  1719 

Unknown  327 


SIZE  AND  TYPE  OF  COMMUNITY 

Rural  203 

Disadvantaged  Urban  391 

Advantaged  Urban  554 

Big  City  313 

Fringe  726 

Medium  City  382 

Small  Places  1373 


Eligible  by 


Grade      Age  &  Grade 


2545 


2545 


Age  or  Grade 
3942 


1193 
i:52 


1193 
1352 


1963 
1979 


182: 
400 
216 
107 


1822 
400 
216 
107 


2756 
639 
388 
159 


632 
482 
705 
726 


632 
482 
705 
726 


905 
843 
1104 
1090 


152 
602 

443 
1170 
175 


152 
602 
443 
1170 
175 


278 
992 
619 
1719 
327 


130 
230 
392 
213 
482 
275 
823 


130 
230 
392 
213 
482 
275 
823 


203 
391 
554 
313 
726 
382 
1373 


365 


Table  16-17 

Numbers  of  Students  in  Bridge  to  1986  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  11/Age  17 


Eligible  by 


TOTAL 
SEX 


Hale 
Female 


Age 
5581 


2660 
2921 


Grade     Age  &  Grade 


5638 


2671 
2967 


4167 


1939 
2228 


Age  or  Grade 
7052 


3392 
3660 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


4077 
888 
420 
196 


4106 
«95 
419 
218 


3178 
591 
264 
134 


5005 
1192 
575 
280 


REGION 

Northeast  1320 

Southeast  1428 

Central  969 

West  1864 

PARENT'S  EDUCATION 

Less  than  High  School  440 

High  School  1284 
Greater  than  High  School  1326 

Graduated  College  2363 

Unknown  150 


1349 
1400 
1054 
1835 


411 
1283 
1341 
2426 

163 


978 
1010 

802 
1377 


242 
916 
1028 
1871 
98 


1691 
1818 
1221 
2322 


609 
1651 
1639 
2918 

215 


SIZE  AND  TYPE  OF  COMMUNITY 

Rural  221 

Disadvantaged  Urban  98 

Advantaged  Urban  1146 

Big  City  1033 

Fringe  943 

Medium  City  649 

Small  Places  1491 


241 
116 
1182 
1033 
942 
670 
1454 


177 
71 
922 
730 
699 
500 
1068 


285 
143 
1406 
1336 
1186 
819 
1877 


366 


Table  16-18 


Nvunbers  of  Students  in  Civics  Bridge 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  13 


TOTAL 


Age 
1938 


Eligible  by 


Grade     Age  &  Grade 


1298 


1298 


Age  or  Grade 
1938 


SEX 


Male 
Female 


988 

950 


594 
704 


594 
704 


988 
950 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


1374 
247 
224 
93 


942 
166 
129 
61 


942 
^66 
129 
61 


1374 
247 
224 
93 


REGION 

Northeast 
Southeast 
Central 
West 


487 
318 
543 
590 


354 
197 
338 

409 


354 
197 
338 
409 


48"" 
3io 
543 
590 


PARENT'S  EDUCATION 

Less  than  High  School  119 

High  School  554 

Greater  than  High  School  325 

Graduated  College  791 

Unknown  142 


64 
358 
233 
567 

72 


64 
358 
233 
567 

72 


119 
554 
325 
791 
142 


SIZE  AND  TYPE  OF  COMMUNITY 

Rural  102 

Disadvantaged  Urban  176 

Advantaged  Urban  317 

Big  City  150 

Fringe  364 

Medium  City  193 

Small  Places  636 


62 
105 
228 
102 
252 
143 
406 


62 

105 
228 
102 
25? 
143 
406 


102 
176 
317 
150 
364 
193 
636 


387 


3.^7 


Table  16-19 


Numbers  of  Students  in  Civics  Bridge 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  17 


Eligible  by 


TOTAL 


SEX 


Male 
Fesaale 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
West 

PARENT'S  EDUCATION 

Less  than  High  School 
High  School 

Greater  than  High  School 
Graduated  College 
Unknovm 

SIZE  AND  TYPE  OF  COMMUNITY 
Rural 

Disadvantaged  Urban 
Advantaged  Urban 
Big  City 
Fringe 
Medium  City 
Small  Places 


Age 
1786 


818 
968 


1263 
310 
142 
71 


378 
459 
337 
612 


154 
427 
434 
725 
40 


91 
32 

331 
348 
303 
226 
455 


Grade     Age  &  Grade 
1333  1333 


591 
742 


992 
199 
91 
51 


281 
321 
279 
452 


100 
291 
328 
590 
20 


73 
23 
268 
245 
236 
164 
324 


591 
742 


992 
199 
9i 
51 


281 
321 
279 
452 


100 
291 
328 
590 
20 


73 
23 
268 
245 
236 
164 
324 


Age  or  Grade 
178S 


818 
968 


1263 
310 
142 
71 


378 
459 
337 
612 


154 
427 
434 
725 
40 


91 
32 
331 
348 
303 
226 
455 


368 


3«8 


ERIC 


Table  16-20 


Numbers  of  Excluded  Students  in  Main  and  Bridge  Samp].es 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  4/Age  9 


TOTAL 


Age 
1480 


Eligible  by 


Grade       Age  &  Grade    Age  or  Grade 


1315 


542 


2253 


SEX 


Male 
Female 


899 
581 


817 
497 


299 
243 


1417 
835 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


506 
308 
567 
99 


410 
243 
537 
125 


133 
83 

282 
44 


783 
468 
822 
180 


ElEGION 

Northeast 
Southeast. 
Central 
West 


269 
344 
200 
667 


195 
288 
138 
694 


82 
66 
44 
350 


382 
566 
294 
1011 


SIZE  AND  TYPE  OF  COMMUNITY 
Rural  67 
Disadvantaged  Urban  304 
Advantaged  Urban  95 
Big  City  305 
Fringe  239 
Medium  City  223 
Small  Places  247 


52 
279 

74 
306 
209 
190 
205 


10 
129 
34 
152 
99 
68 
50 


109 
454 
135 
459 
349 
345 
402 


369 


Ta  le  16-21 


Numbers  of  Excluded  Students  in  Main  and  Bridge  Samples 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  8/Age  13 


Eligible  by 


TOTAL 


1754 


Grade       Age  .Se  Grade    Age  or  Grade 


1852 


542 


3064 


SEX 


Male 
Female 


1125 
579 


1161 
649 


317 
191 


1969 
1037 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
West 


810 
418 
381 
145 


419 
385 
369 
581 


SIZE  AND  TYPE  OF  COMMUNITY 
Rural  89 
Disadvantaged  Urban  331 
Advantaged  Urban  95 
Big  City  268 
Fringe  276 
Medium  City  295 
Small  Places  400 


908 
404 
389 
151 


427 
405 
417 
603 


99 
309 
109 
250 
325 
321 
439 


250 
78 

144 
70 


137 
78 
91 

236 


21 

89 
42 
47 
113 
131 
99 


1468 
744 
626 
226 


709 
712 
695 
948 


167 
551 
162 
471 
488 
485 
740 


ERIC 


370 

390 


Table  16-22 


Numbers  of  Excluded  Students  in  Main  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  12/Age  17 


Eligible  by 


TOTAL 
SEX 


Male 
Female 


Age 

1055 


684 
371 


Grade       Age  &  Grade    Age  or  Grade 


708 


417 
290 


236 


126 
110 


1527 


975 
551 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


476 
300 
214 
65 


366 
207 
95 
40 


119 
68 
35 
14 


723 
439 
274 
91 


REGION 

Northeast 
Southeast 
Central 
West 


201 
287 
225 
342 


SIZE  AND  TYPE  OF  COMMUNITY 
Rural  26 
Disadvantaged  Urban  264 
Advantaged  Urban  41 
Big  City  160 
Fringe  120 
Medium  City  102 
Small  Places  342 


139 
198 
133 
238 


30 
127 

33 
108 

93 

62 
255 


61 
55 
45 
75 


5 
52 
14 
36 
29 
20 
80 


279 
430 
313 
505 


51 
339 

60 
232 
184 
144 
517 


ERIC 


371 


391 


Table  16-23 


Nxunbers  of  Excluded  Students  in  Bridge  Samples 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  11/Age  17 


Eligible  by  — 


TOTAL 
SEX 

Male 
Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


Age 
361 


247 
114 


203 
70 
56 
32 


Grade       A-re  &  Grade    Age  or  Grade 


310 


207 
103 


183 
60 
26 
41 


133 


88 
45 


88 
22 
9 
14 


538 


366 
172 


298 
108 
73 
59 


REGION 

Northeast 
Southeast 
Central 
West 


79 
83 
62 
137 


74 
63 
60 
113 


39 
25 
27 
42 


114 
121 

95 
208 


SIZE  AND  T'PE  OF  COMMUNITY 
Rural  19 
Disadvantaged  Urban  7 
Advantaged  Urban  48 
Big  City  44 
Fringe  84 
Medium  City  59 
Small  Places  100 


15 
15 
50 
22 
52 
67 
89 


8 
4 
3? 
11 
22 
27 
29 


26 
18 
66 
55 

114 
99 

160 


ERIC 


372 

392 


Table  16-24 


Numbers  of  Students  by  Sample  and  Age  Class 


Sample 

Age  Class  9 

Sum  of 

Total  Weights 

Age  Class  13 

Sum  of 
Total  Weights 

Age 
Total 

Class  17 

Sura  of 
Weights 

Main  Assessment* 

Winter  and  spring  combined 
winuer  oniy 
Spring  only 

23012 
12293 
10719 

4477209 
4497709 
4460873 

31601 
16489 
15112 

3737197 
3724549 
3749449 

32710 
18542 
14168 

4024260 
4044142 
3999942 

Civics  Bridge  (to  1976  and  1982)*^ 

1938 

3041846 

1786 

3439421 

Bridge  to  1984^ 

5188 

4477355 

5500 

4312127 

4622 

448/990 

Bridge  to  1986,  Ages  9  and  13^ 

Booklet  91 
oooKiet  7Z 
Booklet  93 

1274 
1240 
1197 

3211177 
3207450 
3210115 

1405 
1281 
1256 

3049468 
3039867 
3068702 

-- 

RrirliTP  ^n   ^QS^f,      Aero  17*^ 

44o/ /44 

Main  and  bridge  combined, 
grade  4/age  9,  grade  8/age  13 

2253 

231907 

3064 

246018 

Main,  grade  12/age  17 

1527 

134353 

BiTidge,  grade  11/age  17 

538 

171553 

*  Grade/age  sample  using  calendar-year  age  definitions 
^  Age-only  sample  using  previous  age  definitions 
^  Grade/age  sanrple  '.sing  previous  ago  definitions 


393 


373 


394 


Table  16-25 


Numbers  of  Students  Assessed  in  Main  Assessment,  Grade  4/Age  9 

(Booklets  1-22) 

  GRADE   

<    4  -    4  >    4  Total 

AGE  <  9 


AGE 


0 

0 

X  ^  7 

WEIGHTED  N 

0 

23915 

0 

23915 

COEFF.  OF  VAR.* 
-  9 

- 

13.90 

- 

13.90 

UNWEIGHTED  N 

5804 

10491 

71 

16366 

WEIGHTED  N 

1204331 

2006376 

13867 

3224573 

STANDARD  ERROR 

8252 

4239 

2393 

9133 

COEFF.  OF  VAR.* 

0.69 

0.21 

17.26 

0.28 

>  9 

UNWEIGHTED  N 

0 

6517 

0 

6117 

WEIGHTED  N 

0 

1228720 

0 

1228720 

STANDARD  ERROR 

8759 

8759 

COEFJ.  OF  VAR.* 

0.71 

0.71 

TOTAL 

UNWEIGHTED  N 

5804 

17137 

71 

23012 

WEIGHTED  N 

1204331 

3259011 

13867 

4477209 

STANDARD  ERROR 

8252 

11015 

2393 

14194 

COEFF.  OF  VAR.* 

0.69 

0.34 

17.26 

0.32 

*  Coeff icle*ir  of  variation  is  defined  as  (100  times  Standard 
Error  divided  by  Weighted  N) . 

374 


Table  16-26 


Numb-irs  of  Students  Assessed  in  Main  Assessment,  Grade  8/Age  13 

(Booklets  1-36) 


<  8 


GRADE 
-  8 


>  8 


Total 


AGE  <  13 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAP.* 


0 
0 


192 
21787 
2368 
10.87 


0 
0 


192 
'5.1787 
2368 
10.87 


AGE  -  13 


UNWEIGHTED  N  7709  14671  91  22471 

WEIGHTED  N  1032989  1601566  16935  2651490 

STANDARD  ERROR  9786  4670  4550  10115 

COEFF.  OF  VAR.*  0.95  0.29  26.87  0.38 


AGE  •>  13 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


0 

0 


8938 
1063920 
7365 
0.69 


0 

0 


8938 
1063920 
7365 
0.69 


AGE  TOTAL 


UNWEIGHTED  N  7709  23801  91  31601 

WEIGHTED  N  1032989  26S7273  16935  3737197 

STANDARD  ERROR  9786  8444  4550  14086 

COEFF.  OF  VAR.*  0.95  0.31  26.87  0.38 


*  Coefficient  of  variation  is  defined  as  (100  tirae'=  Candard 
Error  divided  by  Weighted  N) . 


375 


3PB 


Table  16-27 


Numbers,  of  Students  Assessed  in  Main  Assessment,  Grade  12/Age  17 

(Booklets  1-37) 


<  12 


GRADE 
-  12 


>  12 


Total 


AGE  <  17 


UlIWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


0 
0 


286 
35112 
2823 
8.04 


0 
0 


286 
35112 
2823 
8.04 


AGE  -  17 


UNWEIGHTED  N  839/^  17137 

WEIGHTED  N  1053292  2080066 

STANDARD  ERROR  5236  14474 

COEFF.  OF  VAR.*  0.50  0.70 


0 
0 


25531 
3133358 
15779 
0.50 


AGE  >  17 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


0 
0 


6893 
855789 
28000 
3.27 


0 
0 


6893 
855789 
28000 
3.27 


AGE  TOTAL 


UNWEIGHTED  N  3334  24316 

WEIGHTED  N  105,292  2970968 

STANDARD  ERROR  5236  19439 

COEFF.  OF  VAR.*  0.50  0.65 


0 
0 


32710 
4024260 
20747 
0.52 


*  Coefficient  of  variation  is  defined  as  (100  times  Standard 
Error  divided  by  Weighted  N) . 


376 


ERIC 


Table  16-28 


Ntunbers  of  Students  Assessed  in  Bridge  to  1984,  Grade  A/Age  9 

(Booklets  51-56) 


<  4 


GRADE 
-  4 


>  4 


Total 


AGE  <  9 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


0 
0 


27 
25050 
5112 
20.41 


0 
0 


27 
25050 
5112 
20.41 


AGE  -  9 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


AGE  >  9 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


AGE  TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


1189 
1179250 
17494 
1.48 


0 
0 


1189 
1179250 
17494 
1.48 


2573 
2C04609 
6920 
0.35 


1379 
1251351 
15706 
1.26 


3979 
3281010 
17845 
0.54 


20 

17095 
5230 
30.59 


0 
0 


20 
17095 
5230 
30.59 


3782 
3200954 
20730 
0.65 


1379 
1251351 
15706 
1.26 


5188 
4477355 
31964 
0.71 


*  Coefficient  of  variation  is  defined  as  (100  times  Standard 
Error  divided  by  Weighted  N) . 


393 


Table  16-29 


Numbers  of  Students  Assessed  in  Bridge  to  1984,  Grade  8/Age  13 

(Booklets  51-56) 


AGE  <  13 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


AGE  -  13 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


AGE  >  13 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


AGE  TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


<  8 


0 
0 


1352 
1176048 
19496 
1.66 


0 
0 


1352 
1176048 
19496 
1.66 


GRADE 
-  8 


31 
29904 
7127 
23.83 


2638 
1836364 
7661 
0.42 


1464 
1249074 
12959 
1.04 


4133 
3115341 
17554 
0.56 


>  8 


0 
0 


15 
20738 
15482 
74.66 


0 
0 


15 
20738 
15482 
74. €6 


Total 


31 

29904 
7127 
23.83 


4005 
3033150 
21237 
0.70 


1464 
1249074 
12959 
1.04 


5500 
4312127 
29734 
0.69 


*  Coefficient  of  variation  is  defined  as  (100  times  Standard 
Error  divided  by  Weighted  N) . 


378 


Table  16-30 

Numbers  of  Scudents  Assessed  in  Bridge  to  1984,  Giade  11/Age  17 

(Booklets  51-56) 


<  11 


GRADF 
-  11 


>  11 


"otai 


AGE  <  17 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


0 
0 


399 
370779 
26139 
7.05 


0 
0 


399 
370779 
26139 
7.05 


AGE  -  17 


UNWEi-GKTED  N  623  2694  335  3652 

WEIGHTED  N  806680  2218862  395088  3420630 

STANDARD  ERROR  40941  5164  44128  12266 

COEFF.  OF  VAR.*  5.08  0.23  11.17  0.36 


AGE  >  17 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


571 
696581 
24053 
3.45 


0 

0 


571 
696581 
24053 
3.45 


AGE  TOTAL 


UNWEIGHTED  N  623  3664  335  4622 

WEIGHTED  N  806680  3286221  395088  4487990 

^'TANDARD  ERROR  40941  10847  44128  16310 

COEFF.  OF  VAR.*  5.08  0.33  11.17  0.36 


*  Coefficient  of  variation  is  defined  as  (100  times  Standard 
Error  divided  by  Weighted  N) . 


ERIC 


379 


400 


Table  16-31 

Numbers  of  Students  Assessed  in  Bridge  Co  1986.  Age  9 


AGE  9 


<  4 


GRADE 
~  4 


>  4 


Total 


Booklet  91 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  C  AR.* 


407 
1073424 
49361 
4.60 


863 

2130791 
45194 
2.12 


4 

6961 
4085 
58.68 


1274 
3211177 
22224 
0.69 


Booklet  92 


UNWEIGHTED  N  402  833  5  1240 

WEIGHTED  N  1055792  2142834  8824  3207450 

STANDARD  ERROR  64622  59518  4542  20597 

COEFF.  OF  VAR.*  6.12  2.78  51.47  0.64 


Booklet  93 


UNWEIGHTED  N  387  802  8  1197 

WEIGHTED  N  1087649  2094985  27482  3210115 

STANDARD  ERROR  63643  61277  8101  23096 

COEFF.  OF  VAR.*  5.85  2.92  29.48  0.72 


*  Coefficient  of  variation  is  defined  as  (100  times  Standard 
Error  divided  by  Weighted  N) . 


380 


401 


Table  16-32 


Numbers  of  Students  Assessed  in  Bridge  to  1986,  Age  13 


AGE  13 


<  8 


GRADE 
-  8 


>  8 


Total 


Booklet  91 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


Booklet  92 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


Booklet  93 


486 
1050983 
47648 
4.53 


464 
1160841 
75205 
6.48 


914 
1986691 
49544 
2.49 


805 
1841841 
81211 
4.41 


5 

11795 
6286 
53.29 


12 
37185 
24652 
66.30 


1405 
3049468 
17394 
0.57 


1281 
3039867 
17387 
0.57 


UNWEIGHTED  N  421  826  9  1256 

WEIGHTED  N  1069538  1972011  27153  3068702 

STANDARD  ERROR  78205  80834  16131  16454 

COEFF.  OF  VAR.*  7.31  4.10  59.41  0.54 


*  Coefficient  of  variation  is  defined  as  (100  times  Standard 
Error  divided  by  Weighted  N) . 


381 


402 


Table  16-33 


Numbers  of  Students  Asses  sed  in  Bridge  to  1.986,  Grade  11/Age  17 

(Booklets  61-67) 


AGE  <  17 


<  11 


GRADE 
-  11 


>  11 


Total 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


0 
0 


640 
414914 
30365 
7.32 


0 
0 


640 
414914 
30365 
7.32 


AGE  -  17 


UNWEIGHTED  N  893  4167  521  5581 

WEIGHTED  N  811417  2218503  385821  3415741 

STANDARD  ERROR  37457  4912  39407  15624 

COEFF.  OF  VAR.*  4.62  0.22  10.21  0.46 


AGE  >  17 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


0 
0 


831 
657089 
26348 
4.01 


0 
0 


831 
657089 
26348 
4.01 


AGE  TOTAL 


UNWEIGHTED  N  893  5638  521  7052 

WEIGHTED  N  811417  3290506  385821  4487744 

STANDARD  ERROR  37457  10395  39407  17884 

COEFF.  OF  VAR.*  4.62  0.32  10.21  0.40 


*  Coefficient  of  variation  is  defined  as  (100  times  Standard 
Error  divided  by  Weighted  N) . 


ERIC 


382 


403 


Table  16-34 


Numbers  of  Students  Assessed  in  Civics  Bridge  to  1976  and  1982, 

Age  13  and  Age  17 
(Booklet  90) 


AGE  13 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


  GBADE 

<    8  -  8 

635  1298 

1010355  2024944 

53035  51919 

5.25  2.56 


>    8  Total 

5  1938 

6547  3041846 

4636  16956 

70.81  0.56 


AGE  17 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR.* 


  GRADE 

<  11  -  11 

300  1333 

550974  2624633 

40700  50701 

7.39  1.93 


>  11  Total 

153  1786 

263814  3439421 

28587  9556 

10.84  0.28 


*  Coefficient  of  variation  is  defined  as  (100  times  Standard 
Error  divided  by  Weighted  N) . 


383 


404 


Table  16-35 


Weighted  Percentage  of  Students  in  Main  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  4/Age  9 

  ELIGIBLE  BY   


SEX 

Male 
Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
West 


SIZE  AND  TYPE  OF  COIiMUNITY 
Rural 

Disadvantaged  Urb?.n 
Advantaged  Urban 
Big  City 
Fringe 
Medium  City 
Small  Places 


Age  Grade 


50.38  50.66 
49.62  49.34 


71.08  69.83 

14.51  15.25 

10.69  10.91 

3.72  4.01 


22.50  22.58 

24.96  25.73 

23.95  23.86 

28.60  27.82 


11.31  10.32 

8.27  8.57 

14 .10  14 . 22 

8.81  9.63 

13.40  13.02 

15.36  15.47 

28.74  28.77 


Age  &  Grade     Age  or  Grade 


47.15  52.03 
52.85  47.97 


72.91  69.15 

13.51  15.50 

9.36  11.44 

4.21  3.71 


25.20  21.35 

23.25  26.29 

23.15  24.25 

28.41  28.12 


3.65  5.02 

13.86  14.93 

8.04  7.20 

40.26  36.19 

33.84  36.32 


10.14  11.11 

7.82  8.69 

15.60  13.52 

9,42  9.13 

14.34  12.70 

15.53  15.36 

27.14  29.48 


PARENT'S  EDUCATION 


Less  than  High  School 

4 

36 

4 

82 

High  School 

14 

43 

14 

77 

Greater  than  High  School 

6 

96 

7 

96 

Graduated  College 

36 

86 

38 

04 

Unknown 

37 

08 

34 

.05 

ESTIMATED  XOTAL  POPULATION       3224573         3259011         2006376  4477209 


384 


ERIC 


Table  16-36 


Weighted  Percentage  of  Students  in  Main  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  8/Age  13 


ELIGIBLE  BY 


Age 


Grade       Age  &  Grade     Age  or  Grade 


SEX 


Male 
Female 


49.46 
50.54 


50.17 
49.83 


45.46 
54.54 


51.68 
48.32 


RACE/ETHNICI'fY 
White 
Black 
Hispanic 
Other 


70.86 
14.65 
10.57 
3.92 


70.86 
14.97 
10.13 
4.03 


74.69 
12.88 
8.44 
4.00 


69.22 
15.64 
11.17 
3.96 


REGION 

Northeast  23.08 

Southeai<t  23.75 

Central  25.04 

West  28.13 

PARENT'S  EDUCATION 

Less  than  High  School  7.56 

High  School  26.76 
Greater  than  High  School  17.65 

Graduated  College  38.73 

Unknown  9 . 03 


22.84 
24.46 
24.70 
28.00 


7.92 
26.89 
1G.65 
38.70 

7.57 


24.74 
22.28 
24.98 
28.00 


5.78 
25.48 
19.96 
42.21 

6.32 


22.20 
24.89 
24.82 
28.09 


8.58 
27.40 
17.38 
37.22 

9.14 


SIZE  AND  TYPE  OF  COMMUNITY 

Rural  7.87  7.69  7.20  8.03 

Disadvantaged  Urban  9.35  9.03  7.96  9.71 

.\dvantaged  Urban  9.76  10.21  11.65  9.28 

Big  City  10.59  9.73  9.55  10.42 

Fringe  19.12  19.34  21.46  18.27 

Medium  City  13.76  13.62  13.77  13.65 

Small  Places  29.55  30.38  28.41  30.64 


ESTIMATED  TOTAL  POPULATION 


2651490 


2687273 


1601566 


3737197 


lERJC 


385 


4nG 


Table  16-37 

Weighted  Percentage  of  Students  in  Main  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  12/Age  17 


ELIGIBLE  BY 


Age 


Grade       Age  &  Grade     Age  or  Grade 


SEX 


Male 
Female 


48.32 
51.68 


47.65 
52.35 


43.76 
56.24 


50.18 
49.82 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


73.96 
14.52 
8.10 
3.43 


74.82 
13.41 
7.80 
3.97 


77.46 
12.14 
6.66 
3.74 


72.78 
14.93 
8.62 
3.67 


REGION 

Northeast 
Southeast 
Central 
West 


25.40 
24.03 
24.37 
26.20 


25.63 
23.01 
25.29 
26.07 


27.92 
22.84 
23.74 
25.50 


24.27 
23.89 
25.3 
26.47 


PARENT'S  EDUCATION 


Less  than  High  School 

8 

18 

7 

61 

6 

.28 

8 

74 

High  School 

25 

21 

24 

27 

23 

.13 

25 

59 

Greatier  than  High  School 

24 

04 

24 

17 

25 

.23 

23 

52 

Graduated  College 

39 

80 

41 

43 

43 

.39 

39 

15 

Unknown 

2 

38 

2 

16 

1 

.65 

2 

59 

AND  TYPE  OF  COMMUNITY 

Rural 

4 

81 

4 

80 

4 

.28 

5 

08 

Disadvantaged  Urban 

8 

71 

7 

41 

6 

.67 

8 

81 

Advantaged  Urban 

16 

66 

18 

29 

19 

.21 

16 

54 

Big  City 

12 

66 

12 

.40 

12 

.80 

12 

40 

Fringe 

13 

65 

13 

40 

13 

.53 

13 

53 

Medium  City 

10 

84 

11 

34 

11 

91 

10 

66 

Small  Places 

32 

6(5 

32 

37 

31 

.81 

32 

99 

ESTIMATED  TOTAL  POPULATION 


3133358 


2970968 


2080066 


4024260 


386 


ERIC 


407 


Table  16-38 


Weighted  Percentage  of  Students  in  Bridge  to  1984  Sample 
by  Type  of  Eligibility  and  Subgroup  Clarssif  ication,  Grade  4/Age  9 


E'  TGIBLE  BY 


Age 


Grade       Ape  &  Grade      Age  or  Grade 


SEX 


Male 
Female 


50.27 
49.73 


50.68 
49.32 


45.89 
54.11 


52.53 
47.47 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


71.09 
14.54 
10.72 
3.66 


69.92 
15.22 
10.78 
4.08 


72.96 
13.55 
9.31 
4.17 


69.39 
15.48 
11.39 
3.74 


REGION 

Northeast 
Southeast 
Central 
West 


22.41 
25.32 
23.82 
28.45 


23.41 
26.05 
23.08 
27.45 


25.61 
23.59 
22.31 
28.49 


21.71 
26.63 
23  95 
27.70 


PARENT'S  EDUCATION 

Less  than  High  School 
High  School 

Greater  than  High  School 
Graduated  College 
Unknown 

SIZE  AND  TYPE  OF  COMMUNIfY 
Rural 

Disadvantaged  Urban 
Advantaged  Urban 
B"g  City 
Fringe 
Medium  City 
Small  Places 


56 

5. 

18 

4. 

28 

5.14 

i5 

91 

17 

97 

15. 

78 

17.47 

4 

95 

4 

89 

5. 

12 

4.83 

40 

15 

41 

99 

44. 

13 

39.72 

34 

20 

29 

76 

30 

55 

32.58 

9 

.74 

9 

69 

9 

24 

9.92 

7 

.39 

8 

.30 

6 

63 

8.40 

15 

.71 

13 

.69 

14 

98 

14.56 

7 

.29 

7 

.80 

7 

.71 

7.47 

11 

.77 

11 

.83 

13 

.4C 

11.06 

13 

.91 

14 

.17 

14 

.31 

13.92 

34 

.18 

34 

.52 

33 

.67 

34.66 

ESTIMATED  TOTAL  POPULATION 


3200954 


3281C10 


2004609 


4477355 


ERIC 


387 


408 


Table  16-39 


Weighted  Percentag    of  Students  in  Bridge  to  1984  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  8/Age  13 


ELIGIBLE  BY 


Age  Grade  Age  &  Grade  Age  or  Grade 

SEX 

Male                                   49.52  49.40            43.30  52.08 

Female                                  50.48  50.60            56.70  47.92 


RACE/ETHNICITY 

White  71.18  70.61  74.67  69.28 

Black  14.39  14.90  12.79  15.44 

Hispanic  10.31  10.28  8.40  11.10 

Other  4.12  4.20  4.14  4.17 

REGION 

Northeast  22.75  22.76  24.79  21.89 

Southeast  23.67  23.73  22.15  24.36 

Central  23.64  25.90  2  3.88  25.73 

West  27.94  27.61  27.18  28.02 


PARENT'S  EDUCATION 

Less  ,han  High  School  7.92 

High  School  30.93 

Greater  than  High  School  9.96 

Graduated  College  41.65 

Unknown  9 . 24 


8.45  6.35  8.97 

31.36  30.88  31.26 

10.81  11.25  10.02 

41.05  44.11  40.17 

7.99  7.19  9.21 


SIZE  AND  TYPE  OF  COMMUNITY 


Rural  6.07 

Disadvantaged  Urban  7.33 

Advantaged  Urban  13.30 

Big  City  8.69 

Fringe  14 . 70 

Medium  City  11.41 

Small  Places  38.51 


5.88  6.47  5.76 

7.01  5.76  7.76 

13.87  14.27  13.29 

8.67  8.40  8.80 

14.00  14.63  14.22 

11.49  13.00  10.79 

39.08  37.45  39.38 


ESTIMATED  TOTAL  POPULATION       3033150         3115341         1836364  4312127 


ERIC 


388 


Table  16-40 


Weighted  Percentage  of  Students  in  Bridge  to  1984  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Grade  11/Age  17 


ELIGIBLE  BY 


SEX 


Hale 
Female 


RACE/ETHNICITY 
White 
Black 
Hispanic 
OthctJ- 


Age 


47.85 
52.15 


73.04 
15.08 
8.23 
3.64 


Grade       Age  &  Grade     Age  or  Grade 


46.94 
53.06 


73.65 
14.91 
7.99 
3.44 


45.46 
54.54 


78.17 
12.37 
6.37 
3.09 


48.36 
51.64 


70.96 
16.30 
8.97 
3.77 


REGION 

Northeast 
Southeast 
Central 
West 


22.71 
22.87 
26.31 
28.11 


22.58 
22.04 
27 . 12 
28.25 


23.26 
20  38 
28.06 
28.31 


22.34 
23.50 
26.04 
28.12 


PARENT'S  EDUCATION 

Less  than  High  School  8.92  8.18  6.34  9 . :5 

High  School  30.20  30.39  29.25  30.80 

Greater  than  High  School  17.81  17.70  19.12  17.08 

Graduated  College  40.39  40.91  43.48  39. "5 

Unknown  2.43  2.69  1.68  2.99 


SIZE  AND  TYPE  OF  COMMUNITY 

Rural  7.01  6.95  7.26  6.84 

Disadvantaged  Urbaii  0.83  1.13  0.67  1.13 

Advantaged  Urban  16.00  17.04  17.88  15.84 

Big  City  16.86  17.39  15.79  17.78 

Fringe  14.14  14.04  15.10  13.59 

"edium  Oity  14.87  13.99  14.69  14.32 

Small  Places  30.28  29.46  28.61  30.50 


ESTIMATED  TOTAL  POPULATION       3420630         3286221         2218862  4487990 


ERIC 


389 

410 


Table  16-41 


Weights-d  Percentage  of  Students  in  Bridge  to  1986  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  9 


ELIGIBLE  BY 


4ge 


Grade       Age  &  Grade     Age  or  Grade 


SEX 


Hale 
Female 


49.53 
50.47 


47.03 
52.97 


47.03 
52.97 


49.53 
50.47 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


71.17 
14.47 
10.73 
3.63 


73.29 
13.88 
9.42 
3.42 


73.29 
13.88 
9.42 
3.42 


71.17 
14.47 
10.73 
3.63 


REGION 

Northeast 
Southeast 
Central 
West 


23.82 
25.18 
24.00 
27.00 


29.49 
23.57 
20.60 
26.33 


29.49 
23.57 
20.60 
26.33 


23.82 
25.18 
24.00 
27.00 


PARENT'S  EDUCATION 

Less  than  High  School 
High  School 

Greater  than  High  School 
Graduated  College 
Unknovm 

SIZE  AND  TYPE  OF  COMMUNITY 
Rural 

Disadvantaged  Urban 
Advantaged  Urban 
Big  City 
Fringe 
Medium  City 
Small  Places 


3.69 

3.48 

3. ',8 

3.69 

15.44 

16.06 

16.06 

15.44 

7.57 

8.80 

8.80 

7.57 

42.51 

46.28 

46.28 

42.51 

30.03 

24.94 

24.94 

30.03 

9.60 

8.54 

8.54 

9.60 

7.88 

7.68 

7.68 

7.88 

14.71 

13.50 

13.50 

14.71 

7.01 

7.53 

7.53 

7.01 

11.63 

13.37 

13.37 

11.63 

15.01 

14.82 

14.82 

15.01 

34.16 

34.57 

34.57 

34.16 

ESTIMATED  TOTAL  POPULATION 


9628742 


6368610 


6368610 


9628742 


390 


ERIC 


411 


Table  16-42 


Weighted  Percentage  of  Students  in  Bridge  to  1986  Sample 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  13 


ELIGIBLE  BY 


Grade       Age  &  Grade      Age  or  Grade 


SEX 


Hale 
Female 


50.49 
49.51 


47.27 
52.73 


47.27 
52.73 


50.49 
49.51 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


70.90 
14.59 
10.43 
4.08 


72.59 
14.03 
8.93 
4.45 


72.59 
14.03 
8.93 
4.45 


70.90 
14.59 
10.43 
4.08 


REGION 

Northeast 
Southeast 
Central 
West 


24.11 
23.07 
26.07 
26.75 


26.61 
19.92 
25.52 
27.95 


26.61 
19.92 
25.52 
27.95 


24.11 
23.07 
26.07 
26.75 


PARENT'S  EDUCATION 

Less  than  High  School  7.54  6.27  6.27  7.54 

High  School  25.37  23.60  23.60  25.37 

Greater  than  High  School  15.24  17.37  17.37  15.24 

Graduated  College  43.46  46.01  46.01  43.46 

Unknown  8.18  6.59  6.59  8.18 


SIZE  AND  TYPE  OF  COMMUNITY 


Rural 

5 

01 

5 

07 

5. 

07 

5 

01 

Disadvantaged  Urban 

7 

73 

6 

84 

6. 

84 

7 

73 

Advantaged  Urban 

12 

.64 

13 

65 

13 

65 

12 

.64 

Big  City 

8 

.80 

9 

.39 

9 

39 

8 

.80 

Fringe 

15 

.02 

Id 

.46 

15 

.46 

15 

.02 

Medium  City 

11 

.84 

13 

.67 

13 

.67 

11 

84 

Small  Places 

38 

.96 

35 

.93 

35 

.93 

38 

.96 

ESTIMATED  TOTAL  POPULATION       9158037         5800543         5800543  9158037 


ERIC 


391 

4^2 


Table  16-43 


Weighted  Percentage  of  Students  in  Bridge  to  1986  Sample 
by  Tjrpe  of  Eligibility  and  Subgroup  Classification,  Grade  11/Age  17 


ELIGIBLE  BY 


SEX 


Male 
Female 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


Age 


49.03 
50.97 


73.05 
15.11 
8.23 
3.62 


Grade       Age  &  Grade      Age  or  Grade 


48.65 
5]  .35 


73.56 
14.90 
8.00 
3.54 


47.01 
52.99 


78.19 
12.35 
6.38 
3.08 


49.75 
50.25 


70.88 
16.31 
8.98 
3.83 


REGION 

Northeast  22.74 

Southeast  23.29 

Central  25.75 

West  28.22 

PARENT'S  EDUCATION 

Less  than  High  School  8.83 

High  School  25.21 
Greater  than  High  School  23.76 

Graduated  College  38.98 

Unknown  2.73 


21.95 
21.80 
27.98 
28.27 


7.60 
25.02 
24.04 
39.98 

3.03 


22.42 
20.95 
28.37 
28.^6 


5.53 
23.86 
25.72 
42.30 

2.18 


22.32 
23.35 
26.09 
28.24 


9.56 
25.74 
23.00 
38.08 

3.22 


SIZE  AND  TYPE  OF  C0>5MUNITY 

Rural  5.96  6.77  6.22  6.43 

Disadvantaged  Urban  1.00  1.20  0.96  1.17 

Advantaged  Urban  15.99  16.10  17.18  15.48 

Big  City  17.42  16.56  15.71  17.63 

Fringe  14.55  14.10  14.24  14.37 

Medium  City  13.58  14.67  15.46  13.46 

Small  Places  31.49  30.60  30.24  31.46 


ESTIMATED  TOTAL  POPULiTION       3415741         3290506         2218503  4487744 


ERIC 


392 

413 


Tabl(i  16-44 


Weighted  Percentage  of  Students  in  Civics  Bridge 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  13 


ELIGIBLE  BY 


SEX 


Hale 
Female 


Age 


51.10 
48.90 


Grade       Age  &  Grade      Age  or  Grade 


45.93 
54.07 


45.93 
54.07 


51.10 
48.90 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


71.07 
14.26 
10.54 
4.12 


73.21 
13.90 
8.75 
4.14 


73.21 
13.90 
8.75 
4.14 


71.07 
14.26 
10.54 
4.12 


REGION 

Northeast 
Southeast 
Central 
West 


24.36 
21.57 
27.10 
26.98 


26.72 
20.33 
25.59 
27.35 


26.72 
20.33 
25.59 
27.35 


24.36 
21.57 
27.10 
26.98 


PARENT'S  EDUCATION 

Less  than  High  School  6.21  4.88  4.88  6.21 

High  School  28.65  27.40  27.40  28.65 

Greater  thaa  High  School  16.80  18.23  3  8.23  16.80 

Graduated  College  40.76  43.48  43.48  40.76 

Unknown  7.22  5.69  5.69  7.22 


SIZE  AND  TYPE  OF  COMMUNITY 


Rural 

5 

23 

4 

85 

4 

85 

5. 

23 

Disadvantaged  Urban 

7 

18 

6 

37 

6 

37 

7 

18 

Advantaged  Urban 

15 

64 

16 

64 

16 

64 

15 

64 

Big  City 

9 

79 

9 

87 

9 

87 

9 

79 

Fringe 

14 

20 

14 

26 

14 

26 

14 

.20 

Medium  City 

11 

09 

12 

.57 

12 

57 

11 

09 

Small  Places 

36 

88 

35 

.45 

35 

.45 

36 

.88 

ESTIMATED  TOTAL  POPULATION       3041846         2024944         2024944  3041846 


ERJC 


393 


4M 


Table  16-45 


Weighted  Percentage  of  Students  in  Civics  Bridge 
by  Type  of  Eligibility  and  Subgroup  Classification,  Age  17 

  ELIGIBLE  BY   


SEX 

Hale 
Female 

RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 

REGION 

Northeast 
Southeast 
Central 
West 


SIZE  AND  TYPE  OF  COMMUNITY 
Rural 

Disadvantaged  Urban 
Advantaged  Urban 
Big  City 
Fringe 
Medium  City 
Small  Places 


ESTIMATED  TOTAL  POPULATION 


Age  Grade 


45.90  44.35 
54.10  55.65 


73.03  75.73 

15.12  13.44 

8 . 24  7 . 36 

3.61  3.48 


22.41  21.61 

22.60  20.62 

26.85  30.'  8 

28.14  27.30 


6.86  7.27 

0.91  0.85 

16.35  17.35 

16.85  16.51 

l/'.U  14.19 

14.47  14.27 

30.46  29.55 


Age  &  Grade      Age  or  Grade 


44.35  45.90 
55.65  54.10 


75.73  73.03 

13.44  15.12 

7.36  8.24 

3.48  3.61 


21.61  22.41 

20.62  22.60 
30.48  26.85 
27.30  28.14 


8.03  9.09 

23.30  24.67 

26.19  25.41 

40.97  38.65 

1.27  1.92 


7.27  6.86 

0.85  0.91 

17.35  16.35 

16.51  16.85 

14.19  14.il 

14.27  14.47 

29.55  30.46 


3439421         2624633         2624633  3439421 


PARENT'S  EDUCATION 

Less  than  High  School  9.09  8.03 

High  School  24.67  23.30 

Greater  than  High  School  25.41  26.19 

Graduated  College  38.65  40.97 

Unknown  1.92  1.27 


ERIC 


394 


Table  16-46 


Weighted  Percentage  of  Excluded  Students,  by  Type  of  Eligibility 
and  Subgroup  Classification  in  Grade  4/Age  9  Main  and  Bridge  Samples 


ELIGIBLE  BY 


Grade       Age  &  Grade    Age  or  Grade 


SEX 


Male 
Female 


65.65 
34.35 


66.67 
33.25 


61.88 
38.12 


66.83 
33.13 


RACE/ETHNICITY 
Jhite 
Black 
Hispanic 
Other 


53.56 
24.46 
16.98 
5.01 


51.38 
22.01 
18.73 
7.88 


45.73 
19.26 
26.25 
8.76 


53.73 
24.04 
16.34 
5.90 


REGION 

Northeest 
Southeast 
Central 
West 


21.44 
26.81 
21.76 
30.00 


19.73 
26.18 
16.77 
37.32 


20.42 
15.47 
15.11 
47.99 


20.71 
28.22 
20.24 
30.84 


SIZE  AND  TYPE  OF  COMMUNITY 
Rural 

Disadvantaged  Urban 
Advantaged  Urban 
Big  City 
Fringe 
Medium  City 
Small  Places 


7 

08 

7. 

44 

3. 

96 

7. 

79 

13 

82 

12 

88 

14. 

01 

13 

29 

8 

42 

8 

94 

10 

00 

8 

43 

14 

67 

16 

61 

19 

21 

14 

93 

14 

.63 

14 

89 

20 

53 

13 

77 

15 

.19 

15 

.73 

15 

72 

15 

.39 

26 

.21 

23 

.51 

16 

.56 

26 

.41 

ESTIMATED  TOTAL  POPULATION      148361  22555 


59008 


231907 


ERIC 


395 


4t6 


Table  16-47 


Weighted  Percentage  of  Excluded  Students,  by  Type  of  Eligibility 
and  Subgroup  Classification  in  Grade  8/Age  13  Main  and  Bridge  Samples 


Age 


SEX 

Male  66.24 

Female  31.64 

RACE/ETHNICITY 

White  52.93 

Black  25.65 

Hispanic  15 . 12 

Other  6.30 

REGION 

Northeast  27.07 

Southeast  22.64 

Central  21.42 

West  28.87 

SIZE  AND  TYPE  OF  COMMUNITY 

Rural  6.43 

Disadvantaged  Urban  16.82 

Advantaged  Urban  6.21 

Big  City  16.76 

Fringe  13.10 

Meditim  City  14.35 

Small  Places  26.33 


  ELIGIBLE  BY   

Grade     Ape  &  Grade    Age  or  Grade 


63.81  60.22  65.77 

34.44  34.44  32.80 


57.84  55.74  55.28 

21.10  14.84  24.72 

14.64  18.71  14.31 

6.41  10.71  5.69 


25.78  27.14  26.33 

21.45  14.23  23.24 

23.77  18.62  23.18 

28.99  40.01  27.25 


7.81  4.56  7.49 

14.08  13.31  15.80 

6.11  8.94  5.74 

12.12  7.46  15.55 

16.67  20.46  14.01 

14.92  22.42  13.45 

28.29  22.85  27.97 


ESTIMATED  TOTAL  POPULATION    144182       139191  37354  246018 


396 

er|c  ^  ^  '7 


Table  16-48 


Weighted  Percentage  of  Excluded  Students,  by  Type  of  Eligibility 
and  Subgroup  Classification  in  Grade  12/Age  17  Main  Sample 


ELIGIBLE  BY 


Age 


Grade     Age  &  Grade    Age  or  Grade 


SEX 


Male 

C  0 
D  J 

C  0 

oz 

^"7 
j/  . 

CI 

ji . 

77 

fi9 
OZ  . 

A  7 

reniaxe 

JO 

J  o 

A  9 

4S 

HO  . 

£.  'J 

/I7TWMTPTTV 

wnx  ue 

J** 

fin 

flZi 

£.  £. 

94 

111  f^^i^ 

ZD 

1  J 

Z  J  . 

/■ 

't  J 

ZU 

9S 
Z  J  . 

91 
Z  X 

Hispanic 

li 

"7  A 

R  Q 

jy 

o 

o . 

0  J 

1  9 
IZ  . 

Other 

6 

21 

6, 

94 

7. 

95 

6. 

27 

}N 

Northeast 

25 

.83 

24 

33 

32 

34 

24 

01 

Southeast 

25 

.64 

25 

63 

20 

88 

26 

44 

Central 

22 

.94 

20 

.06 

22 

29 

21 

65 

West 

25 

.59 

29 

98 

24 

49 

27 

90 

AND  TYPE  OF  COMMUNITY 

Rural 

3 

.02 

4 

.92 

1 

.75 

4 

.16 

Disadvantaged  Urban 

19 

.23 

I'f 

.38 

17 

.53 

17 

.17 

Advantaged  Urban 

5 

.18 

.18 

7 

.01 

5 

.35 

Big  City 

10 

.21 

9 

.77 

9 

.13 

10 

.18 

Fringe 

13 

.25 

13 

75 

14 

.09 

13 

.35 

Medium  City 

9 

.95 

8 

.71 

9 

.38 

9 

.44 

Small  Places 

39 

.16 

42 

.29 

41 

.10 

40 

.35 

ESTIMATED  TOTAL  POPULATION 


91954 


64970 


22571 


134353 


ERIC 


397 


4tS 


Table  16-49 


Weighted  Percentage  of  Excluded  Students,  by  Tjrpe  of  Eligibility 
and  Sutgroup  Classification  in  Grade  11/Age  17  Bridge  Sample 


ELIGIBLE  BY 


Age 


Grade     Ape  &  Grade    Age  or  Grade 


SEX 


Male 
Female 


69.45 
30.55 


67.67 
32.33 


65.99 
34.01 


69.13 
30.87 


RACE/ETHNICITY 
White 
Black 
Hispanic 
Other 


64.83 
16.47 
13.61 
5.10 


64.36 
17.10 
8.22 
10.32 


71.56 
13.26 
7.71 
7.47 


63.32 
17.41 
11.79 
7.49 


REGION 


Northeast 

21 

99 

21 

88 

33 

17 

19 

84 

Southeast 

21 

72 

17 

67 

17 

18 

20 

37 

Central 

28 

36 

32 

62 

26 

27 

31 

07 

West 

27 

92 

27 

83 

23 

38 

28 

72 

AND  TYPE  OF  CCIitlUNITY 

Rural 

10 

64 

8 

45 

11 

09 

9 

36 

Disadvantaged  Urban 

1 

04 

2 

90 

1 

70 

1 

93 

Advantaged  Urban 

10 

94 

13 

29 

21 

97 

10 

15 

Big  City 

7. 

29 

3 

54 

4 

18 

5 

84 

Fringe 

16. 

02 

9 

56 

10 

14 

13. 

61 

Medium  City 

19. 

53 

22 

95 

23 

60 

20. 

62 

Small  Places 

34. 

54 

39 

30 

27 

32 

38. 

48 

E£  -IMATED  TOTAL  POPULATION 


110521 


93092 


32060 


171553 


ERIC 


398 

419 


Table  16-50 


Niv"abers  of  Students  in  Main  Sample  with  Proficiency  Scores 
by  lype  of  Eligibility,  Grade  4/Age  9 


ELIGIBLE  BY 


Subject  Area /Sample 

Age 

Grade  Age 

&  Grade  Ape  or  ( 

Refldinp  Fociic;pd-BTB 

4423 

4534 

2780 

6177 

Reading  Intercorrelation 

1882 

1957 

1201 

2638 

Writing  Focused-BIB 

and  Long  Writing 

6331 

6679 

4129 

8881 

Civics  Focused-BIB 

1858 

1974 

1180 

2652 

Civics  Intercorrelation 

1882 

1957 

1201 

2638 

History  Focused-BIB 

1872 

1993 

1201 

2664 

History  Intercorrelation 

1882 

1957 

1201 

2638 

Geography  Focused-BIF 

0 

0 

0 

0 

Geography  Intercorrelation 

0 

0 

0 

0 

TOTAL 

16366 

17137 

10491 

23012 

399 


Table  16-51 


Ntimbers  of  Students  in  Main  Sample  with  Proficiency  Scores 
by  Type  of  Eligibility,  Grade  8/Age  13 


ELIGIBLE  BY 


Subject  Area/Sample 

Age 

Grade  Age 

&  Grade 

Age  or 

Reading  Focused- BIB 

4250 

4404 

2742 

5912 

Reading  Intercorrelation 

1819 

1943 

1172 

2590 

Writing  Focused- BIB 

and  Long  Writing 

6059 

6525 

3987 

8597 

Civics  Focused-BIB 

4249 

4487 

2755 

5981 

Civics  Intercorrelation 

1819 

1943 

1172 

2590 

History  Focused-BIB 

4303 

4519 

2834 

5988 

History  Intercorrelation 

1819 

1943 

1172 

2590 

'Jeography  Focused-BIB 

0 

0 

0 

0 

Geography  Intercorrelation 

0 

0 

0 

0 

TOTAL 

20680 

21878 

13490 

29068 

ERIC 


400 

421 


Table  16-52 


unbers  of  Students  In  Main  Sample  with  Proficiency  Scores 
by  Type  of  Eligibility,  Grade  12/Age  17 


ELIGIBLE  BY 


Subject  Area/Sample 

Reading  Focused-BIB 
Reading  Intercorrelatlon 

Uriting  Focused-BIB 
and  Long  Writing 

Civics  Focused-BIB 
Civics  Intercorrelatlon 

History  Focused-BIB 
History  Intercorrelatlon 

Geography  Focused-BIB 
Geography  Intercorrelatlon 

TCTAL 


Age 

4509 
1879 


Grade  Ape  &  Grade  Ape  or  Grade 


4250 
1851 


2991 
1292 


5768 
2438 


6360 

6069 

4259 

8170 

4416 

4275 

3008 

5683 

1249 

1233 

861 

1621 

4561 

4268 

3049 

5780 

1266 

1239 

873 

1632 

1912 

1800 

1266 

2446 

1243 

1230 

850 

1623 

23637 

22513 

15865 

30285 

i 


ERJC 


401 


4P2 


Table  16-53 


Numbers  of  Students  In  Bridge  to  1984  Sample  with  Proficiency  Scores 
by  Type  of  Eligibility,  Grade  4/Age  9 

  ELIGIBLE  BY   

Subject  Area  Age  Grade  Age  &  Grade  k^e  or  Grade 


Reading 

3782 

3979 

2573 

5188 

Wrlcing 

2154 

3327 

2154 

3327 

TOTAL 

3782 

3979 

2573 

5188 

ERIC 


402 

4?3 


Table  16 -54 


Numbers  of  Students  in  Bridge  to  198A  Sample  with  Proficiency  Scores 
by  Type  of  Eligibility,  Grade  8/Age  13 

  ELIGIBLE  BY   

Subject  Area  Age  Grade  Aee  &e  Grade  Age  or  Gradfe 


Reading 

A005 

A133 

2f,38 

5500 

WrjLCing 

2638 

A133 

2638 

4133 

TOTAL 

A005 

A133 

2638 

5500 

Table  16-55 


Numbers  of  Students  in  Bridge  to  1984  Sample  with  Proficiency  Scores 
by  Type  of  Eligibility,  Grade  11/Age  17 


ELIGIBLE  BY 


Subiect  Area 

Age 

Grade  Age  &  Grade  Ag< 

i  or  Grade 

Reading 

3652 

3664 

2694 

4622 

Writing 

2694 

3664 

2694 

.■564 

TOTAL 

3652 

3664 

2694 

4622 

404 


Table  16-56 


Ntunbers  of  Students  in  Bridge  to  1986  Sample  with  Proficiency  Scores 

by  Type  of  Eligibility,  Age  9 

  ELIGIBLE  BY   

Subject  Area  Age  Grade  Age  &  Grade  Age  or  Grade 


Reading 

3711 

2498 

2498 

3711 

Mathematics 

3711 

2498 

2498 

3711 

Science 

3711 

2498 

2498 

3711 

TOTAL 

3711 

2498 

2498 

3711 

405 


4P6 


Table  16-57 


Numbers  of  St'^dents  In  Bridge  to  1986  Sample  with  Proficiency  Scores 
by  Type  of  Eligibility,  Age  13 


EUGIBLE  BY 


Subiect  Area 

Age 

Grade  Age 

6c  Grade 

Age  or  ( 

Reading 

3942 

2545 

2545 

3942 

Mathematics 

3942 

2545 

2545 

3942 

Science 

3942 

2545 

2545 

3942 

TOTAL 

3942 

2545 

2545 

3942 

406  4?7 


Table  16-58 


Ntunbers  of  Students  in  Bridge  to  1986  Sample  with  Proficiency  Scores 
by  Type  of  Eligibility,  Grade  11/Age  17 


  ELIGIBLE  BY   

Subject  Area  Age  Grade  Ape  &  Grade  Age  or  Grade 


Reading 

0 

0 

0 

0 

History 

0 

0 

0 

0 

Mathematics 

1852 

1360 

1360 

1852 

Science 

1862 

1389 

1389 

ld62 

TOTAL 

3105 

2304 

2304 

3105 

407 


4?S 


Table  16-59 

Numbers  of  Students  in  Civics  Bridge  with  Proficiency  Scores 
by  Type  of  Eligibility,  Age  13 

  E'.IGIBLE  BY   

Subject  Area  Age  Grade  Age  &  Grade  Age  or  Grade 

Civics  1938  1298  1298  1938 

TOTAL  1938  1298  1298  1938 


lERlC 


408 


4P9 


Table  16-60 


Numbers  of  Students  in  Civics  Bridge  with  Proficiency  Scores 
by  Type  of  Eligibility,  Age  17 

  ELIGIBLE  BY   

Subject  Area  Age  Grade  Age  &  Grade  Age  or  Grade 

Civics  1786  1333  1333  1786 

TOTAI.  1786  1333  1333  1786 


409 


430 


Table  16-61 


Weighted  jfr^ificiency  Means,  Standard  Devlatlotis,  and  Percentiles 

with  Standard  Errors 
for  Main  Focused-BIB  Reading  Samples,  by  Grade  and  Subgroup 


Grade  4 


Grade  8 


Grade  12 


TOTAL  SAMPLE 

Mean  230.4 

Standard  Deviation  41.4 
Percentiles 

5  156.4 

10  174.2 

25  204.3 

50  233.6 

75  259.0 

90  280.3 

95  293.2 

MALE  STUDENTS 

Mean  226.6 

Standard  Deviation  43.6 
Percentiles 

5  148 . 7 

10  166.5 

25  198.7 

50  229.5 

75  257.1 

90  279.3 

95  293.2 

FEMALE  STUDENTS 

Mean  234.4 

Standard  Deviation  38.5 
Percentiles 

5  167.2 

10  182.8 

25  210.1 

50  237.6 

75  260.6 

90  281.3 

95  292.9 


1.1 
0.7 

1.3 
2.3 
2.0 
1.2 
1.1 
0.9 
1.2 


1.5 
1.0 

2.5 
4.4 
2.7 
1.4 
2.2 
1.9 
2.3 


1.2 
0.6 

2.0 
1.5 
1.0 
1.1 
1.1 
1.7 
1.6 


262.8 
37.3 

195.0 
211.6 
239.8 
266.0 
288.6 
308.2 
320.1 


256.4 
39.4 

186.0 
201.0 
230.7 
259.9 
284.2 
304.7 
317.3 


269.6 
33.8 

211.3 
225.0 
248.5 
271.6 
292.2 
310.6 
322.4 


1.0 
0.6 

2.6 
1.4 
1.6 
1.1 
1.7 
1.0 
1.8 


1.3 
0.8 

3.7 
1.9 
1.9 
2.1 
1.7 
1.9 
1.8 


1. 
0. 

1. 
1. 
1. 
1.2 
1.0 
1.2 
2.2 


287.1  ( 
34.8  ( 

224.9 
241.1 
265.9 
290.0 
310.8 
329.1 
340.0 


282.9 
36.8 

214.6 
232.8 
259.7 
286.2 
307.9 
327.3 
338.1 


291.0 
32.4 

235.6 
249.3 
270.6 
292.8 
313.2 
330.5 
341.3 


0.8) 
0.6) 

2.4) 
1.4) 
1.3) 
1.0) 
1.0) 
1.3) 
1.8) 


1.1) 
0.8) 

2.3) 
1.9) 
2.5) 
1.5) 
1.4) 
0.9) 
2.2) 


1.1) 
0.8) 

4.2) 
1.5) 
1.4) 
1.3) 
1.2) 
1.7) 
3.2) 


ERIC 


410 

431 


Table  16-61  (continued) 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  EiTrors 
for  Main  Focused-BIB  Reading  Samples,  by  Grade  and  Subgroup 


Grade  4  Grade  8  Grade  12 


WHITE  STUDENTS 


Mean 

238. 

1 

(  1.2) 

269 . 

2  ( 

1. 

3) 

292. 

6  ( 

1.0) 

Standard  Deviation 

39. 

3 

(  0.8) 

36 . 

2  ( 

0. 

7) 

33. ( 

0.6) 

Percentiles 

5 

167. 

1 

I  2.9) 

202. 

3  ( 

5. 

7) 

233. 

3  ( 

:  3.6) 

10 

186. 

0 

(  3.6, 

219. 

8  ( 

2. 

7) 

249. 

2  ( 

:  1.9) 

25 

213. 

9 

(  1.3) 

248 . 

1  ( 

1. 

7) 

272. 

6  ( 

:  1.2) 

50 

240. 

8 

(  1.7) 

272. 

6  ( 

1. 

3) 

one 

295 , 

1  i 

:  1.0) 

75 

264. 

4 

(  1.1) 

293. 

6  ( 

0. 

7) 

315 , 

3  i 

:  1.3) 

90 

285 . 

2 

(  1.6) 

312 , 

6  1 

1  . 

3) 

Q  Q  0 

J  JZ  . 

C  i 

D  ' 

,  0.7) 

95 

297 . 

8 

(  2.7) 

323 , 

7  < 

.     1  « 

5) 

O  A  O 

343 , 

6  1 

.  2.2) 

BIACK  STUDENTS 

Mean 

210, 

6 

(  1.9) 

245 

"7  t 

1  I 

,  2 

0) 

270, 

•< 

,  1.6) 

Scandard  Deviation 

39 , 

1 

(  1.0) 

32 

8  ( 

.  1 

2) 

31 

1  1 

,  1.2) 

Percentiles 

5 

143, 

0 

(  4.2) 

187 

4  ( 

[  4 

0) 

213 

9 

[  3.5) 

10 

158 

4 

(  4.7) 

201 

2  1 

.  3 

7) 

ZZo 

Q 
O 

[  3.2) 

25 

184 

7 

(  3.4) 

225 

5  i 

,  2 

3) 

250 

3 

C  3.5) 

jU 

on  o 

u 

O/i  ft 

^  z 

.u; 

272 

9 

75 

238 

8 

(  2.0) 

267 

.0  < 

;  2 

8) 

292 

.0 

C  1.7) 

90 

258 

0 

(  1.9) 

284 

.9  ( 

;  2 

.6) 

307 

.1 

(  2.5) 

95 

270 

5 

(  3.4) 

297 

.2  ( 

;  3 

.8) 

317 

.5 

(  4.4) 

HISPANIC  STUDENTS 

Mean 

209 

.9 

(  2.4) 

243 

.6 

[  2 

.1) 

267 

.1 

(  2.4) 

Standard  Deviation 

41 

.5 

(  1.3) 

37 

.6 

[  1 

.3) 

35 

.7 

(  1.5) 

Percentiles 

5 

134 

.8 

(  7.6) 

177 

.6 

.3) 

200 

.8 

(  7  0) 

10 

153 

.2 

(10.4) 

190 

.9 

.3) 

218 

.1 

(  3.3) 

25 

182 

.5 

(  4.0) 

217 

.6 

(  4 

.3) 

244 

.3 

(  4.3) 

50 

213 

.4 

(  1.9) 

246 

.9 

(  3 

.4) 

269 

.9 

(  2.3) 

75 

240 

.3 

(  2.9) 

271 

.5 

(  2 

.5) 

292 

.7 

(  1.9) 

90 

260 

.1 

(  6.4) 

288 

.2 

(  2 

.5) 

310 

.8 

(  6.5) 

95 

272 

.0 

(  4.1) 

300 

.6 

(  3 

.1) 

321 

.0 

(  2.4) 

Table  16-62 

Weighted  Proficiency  rieans,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Bridge  to  1984  Reading  Samples,  by  Age  and  Subgroup 


Age  9 


Age  13 


AgeJJ 


TOTAL  SAMPLE 

Mean  211 . 8 

Standard  Deviation  41.2 
Percentiles 

5  142.0 

10  156.8 

25  184.3 

50  213.7 

75  240.1 

90  263.0 

95  277.5 

MALE  STUDENTS 

Mean  207 . 5 

Standard  Deviation  42.7 
Percentiles 

5  136.6 

10  151.1 

25  178.4 

50  209.8 

75  237.1 

90  260.4 

95  275.1 

FEMALE  STUDENTS 

Mean  216.3 

Standard  Deviation  39.2 
Percentiles 

5  149.4 

10  164.4 

25  190.6 

50  217.3 

75  242.6 

90  265.3 

95  279.2 


1.2 
1.0 


1.5 
1.1 

2.9 
2.3 
1.7 
1.8 
1.7 
2.0 
2.3 


1.4 
1.1 

5.2 
4.8 
2.4 
i.9 
1.0 
2.1 
3.3 


257.5 
34.7 

199.7 
213.0 
234.3 
257.9 
281.4 
301.6 
313.7 


251.8 
35.3 

192.7 
206.8 
227.8 
252.2 
276.5 
297.2 
309.4 


263.0 
33.1 

207.4 
221.1 
240.1 
263.0 
285.8 
305.2 
317.7 


0.9 
0.4 

1.6 
1.2 


1.2 
0.6 

2.7 
1.6 
2.0 
1.1 
2.0 
1.5 
2.8 


1.0 
0.5 

3.8 
1.4 
1.6 
1.3 
1.0 
1.1 
3.2 


290.1 
37.1 

226.2 
241.5 
265.8 
291.1 
316.0 
336.9 
348.7 


286.0 
37.5 

222.0 
236.3 
261.6 
287.0 
312.0 
333.4 
345.6 


293.8 
36.3 

231.8 
246.6 
270.2 
294.6 
319.4 
339.8 
351.7 


412 


433 


Table  16-62  (continued) 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Bridge  co  1984  Reading  Samples,  by  Age  and  Subgroup 


Age  9 


Age  17 


WHITE  STUDENTS 


Mean 

217. 

7 

( 

1.5) 

261. 

3  ( 

1. 

0) 

294. 

7 

( 

1. 

3) 

Standard  Deviation 

39. 

3 

( 

1.0) 

33. 

9  ( 

0. 

5) 

36. 

0 

( 

0. 

7) 

Percentiles 

5 

150. 

4 

( 

3.4) 

204. 

2  ( 

1. 

4) 

232. 

7 

( 

1. 

0) 

10 

165. 

1 

( 

3.9) 

217. 

2  ( 

1. 

9) 

247. 

4 

( 

3. 

8) 

25 

191. 

8 

( 

2.4) 

238. 

4  ( 

1. 

0) 

271. 

4 

( 

1. 

7) 

50 

219. 

1 

( 

1.2) 

262. 

2  ( 

1. 

1) 

295. 

4 

( 

1. 

6) 

75 

244. 

3 

( 

1.8) 

285. 

1  ( 

0. 

9) 

319. 

9 

( 

1. 

9) 

90 

266. 

8 

( 

2.2) 

304. 

2  ( 

1. 

4) 

339. 

8 

( 

1. 

5) 

95 

280. 

6 

( 

2.5) 

315. 

8  ( 

:  1- 

1) 

351, 

6 

( 

2. 

9) 

BLACK  STUDENTS 

Mean 

188, 

5 

( 

2.6) 

242. 

9  < 

:  2. 

3) 

274 

4 

( 

2, 

6) 

Standard  Deviation 

39 

4 

( 

1.5) 

32. 

1  ( 

:  1. 

3) 

35 

9 

( 

1, 

3) 

Percentiles 

5 

124 

7 

( 

6.3) 

190, 

6  ( 

;  3, 

1) 

214 

5 

( 

9 

5) 

10 

138 

.3 

( 

3.3) 

202, 

2  ( 

;  3 

1) 

227 

.8 

( 

4 

.2) 

25 

161 

.8 

( 

2.3) 

222, 

0  ( 

[  2 

4) 

250 

.5 

( 

2 

3) 

50 

188 

.3 

( 

3.9) 

242 

4  < 

[  2 

7) 

274 

.3 

( 

3 

j) 

75 

216 

.5 

( 

2.8) 

263 

[  4 

4) 

299 

/■ 

( 

3 

.0) 

90 

238 

.2 

( 

3.7) 

283 

.6  1 

[  ^ 

.7) 

321 

.0 

( 

3 

.8) 

95 

252 

.2 

4.3) 

298 

9  ( 

[  2 

.1) 

333 

.1 

( 

4 

.8) 

HISPANIC  STUDENTS 

Mean 

193 

.7 

( 

3.9) 

240 

.1 

(  3 

.5) 

270 

.8 

( 

4 

.0) 

Standard  Deviation 

41 

.5 

( 

2.6) 

34 

.6 

(  2 

.2) 

37 

.7 

( 

2 

.0) 

Percentiles 

5 

121 

.9 

( 

10.8) 

181 

.7 

(  8 

.7) 

204 

.2 

( 

LI 

.5) 

10 

140 

.3 

( 

7.3) 

194 

.6 

(  3 

.7) 

218 

.0 

( 

6 

.9) 

25 

164 

.9 

( 

5.0) 

213 

.9 

(  6 

.0) 

246 

.4 

( 

5 

.5) 

50 

196 

.0 

( 

3.3) 

240 

.3 

(  3 

.9) 

273 

.6 

( 

5 

.0) 

75 

222 

.0 

( 

6.0) 

262 

.0 

(  5 

.2) 

297 

.9 

( 

7 

.0) 

90 

246 

.7 

( 

7.9) 

284 

.0 

(  8 

.4) 

315 

.9 

( 

18 

.0) 

95 

258 

.6 

( 

11.3) 

297 

.3 

(  9 

.9) 

328 

.0 

( 

8 

.6) 

ERIC 
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Table  16-63 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Main  Writing  Samples,  by  Grade  and  Subgroup 


Grade  4  Grade  8  Grade  12 

TOTAL  SAMPLE 


Mean 

Q 

(  1 

.0) 

zuy 

,  3 

U 

.  y  ^ 

OO  A 

ZZ4 

.  z 

( 

1 

.3) 

ouiinaaira  ue via t Aon 

Q 
.  O 

(  0 

.7) 

A  Q 

Q 
.  O 

f\ 

u 

AO 

49 

»  Z 

( 

A 

0 

.8) 

irercenui  xes 

O 

(  1 

.2) 

13/ 

.  1 

o 
z 

7  \ 
•  /  ) 

1  A  O 

14z 

0 
.  3 

( 

A 

2 

.0) 

T  n 
xu 

loo 

Q 

(  1 

.3) 

i33 

"7 

/ 

1 

O  \ 

.  z) 

161 

.  1 

( 

A 

2 

.0) 

(  1 

.0) 

ioU 

Q 
3 

1 

.  i; 

ly  1 

.  y 

( 

1 

•  5) 

50 

192 

.0 

(  1 

.3) 

209 

9 

( 

1 

.3) 

224 

.9 

( 

1 

.3) 

75 

219 

.7 

(  1 

.2) 

238 

9 

( 

1 

.5) 

257 

.4 

( 

1 

.9) 

VKJ 

OA  Q 

(  1.4) 

O/I  A 
ZD^f 

1 

.  3  ) 

o  o  ^ 

zoo 

.4 

( 

A 

2 

.6) 

ZoU 

A 

0 

z 

,  u  ) 

3U3 

►  / 

( 

A 
Z 

7  \ 

MAT  P  CTTinPMTC 

A 
.  H- 

(  1 

3) 

zuu 

Q 

y 

1 

A  ^ 

oi  0 
ZIZ 

3 

( 

1 

A  \ 

.4) 

ocaiiuaLU  i/6viation 

A 
'+ 

(  0 

9) 

A 
H 

n 
u 

Q  \ 
,  y ) 

A  Q 

c 
0 

/ 
\ 

1 

.1) 

X  X  J 

3 

(  2 

7) 

izy 

3 

A 

0  ^ 
z  ^ 

131 

7 

/ 
( 

z 

If) 

uu 

z 

(  2 

0) 

l'*3 

O 

0 

z 

0  ^ 

131 

A 

U 

/ 
\ 

1 

0) 

13  / 

3 

(  1 

6) 

1  70 
Liz. 

1 

* 

1 

3  ) 

loU 

/: 
0 

/ 
( 

1 

50 

186 

1 

f  1 

1) 

901 

1 

X 

.J  J 

7 

1 

1 

3; 

75 

212 

9 

[  1 

7) 

230. 

4 

1 

4) 

245 

5 

( 

1 

7) 

90 

236 

5 

[  1 

8) 

255. 

8 

2 

3) 

273 

2 

( 

2 

.3) 

95 

249 

8 

[  1 

8) 

271. 

1 

2 

5) 

290 

4 

( 

3 

.7) 

FEMALE  STUDENTS 

Mean 

197 

;  1 

3) 

218. 

5 

/ 
\ 

1 

2) 

234. 

8 

( 

1 

8) 

Standard  Deviation 

42 

2  ( 

:  G. 

7) 

42. 

4 

( 

0 

8) 

47. 

3 

( 

1 

1) 

Percentiles 

5 

127. 

2  ( 

:  2. 

6) 

148. 

8 

( 

2. 

2) 

156. 

0 

( 

3 

3) 

10 

143. 

0  < 

;  i. 

6) 

164. 

5 

( 

1. 

5) 

174. 

7 

( 

2. 

5) 

25 

169. 

9  ( 

:  1. 

5) 

190. 

2 

( 

1. 

3) 

203. 

4 

( 

1. 

1) 

50 

198. 

1  ( 

:  1. 

5) 

219. 

0 

( 

1. 

8) 

235. 

6 

( 

2. 

8) 

75 

226. 

3  ( 

:  1. 

7) 

247. 

2 

( 

1. 

6) 

267. 

1 

( 

1. 

9) 

90 

251. 

0  ( 

:  1. 

6) 

272. 

1 

( 

2. 

3) 

295. 

2 

( 

4. 

0) 

95 

265.4  ( 

:  4. 

7) 

287. 

7 

( 

4. 

4) 

311. 

5 

( 

5. 

2) 

414 
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Table  16-63  (continued) 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Main  Writing  Samples,  by  Grade  and  Subgroup 


WHITE  STUDENTS 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 

BLACK  STUDENTS 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 

HISPANIC  STUDENT^ 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 


Grade  4 

Grade 

8 

Grade  12 

197.6 

(  1.3) 

o  1     A  / 
Zlb.U  V 

230.5 

(  1.5) 

/    A    "7  \ 

42.2  ( 

0.7) 

A  7  0 

^  JL .  JL ; 

1  OA  C 

(  z .  1) 

14 J. y  V 

J  .  z) 

IDU .  o 

/    9    ft  N 

^  z .  o  ; 

1  A  ^  A 

\  1.4; 

1           A  / 

1     "7  \ 
1  .  /  ) 

JL07  .  D 

\    ^3  .  Z  J 

1  "71  "7 
1/1.  / 

\  1 .  ^) 

loo.l  ( 

1     Q  \ 

L»  J) 

1  00  R 

\  JL  .  /  ; 

1  no  1 
19o .  1 

(  1 .  o) 

ZlD.4  V 

1     A  \ 

1 .4^ 

Z  JJL  .  H 

(  ^  9^ 
\  J. .  z  ; 

224.8 

(  2.0) 

O  A  A     Q  / 
Z44    O  ( 

1.1; 

262.9 

(  1.7) 

O  /.  Q  A 

(  z .  / ; 

1  .  O/ 

Z^U  .  J 

\   -3  .  v; 

261.8 

(  1.9) 

284.2  ( 

2.8) 

307.4 

(  2.1) 

168.8 

(  1.9) 

lo / . J  \ 

z .  z; 

200.7 

(  2.3) 

JL 

/  1  1  ^ 

41.3  ( 

1.3) 

AS  7 

^     X  .  O  J 

\  ^  •  ^) 

A  1  ^ 
H .  JL ; 

XZ^  .  J 

1         .  U 

(  0\ 

TO/:/,  / 

J  •  D  ) 

lZt9  1 

^'  O'^ 

1  AT  ^ 

\    L  •  J  ) 

.0  ^ 

0  ft  ^ 
z .  0  ; 

1  Q 

107  .  0 

\    L  »  L) 

1         ft  ^ 
JLoO .  o  ^ 

J .  0  ; 

900  L 

196  8 

(  2.4) 

91  '^    ft  / 
Z  JL  J  .0  ^ 

7  ^ 

231.9 

(  3.1) 

221.5 

(  3.4) 

OOQ     Q  / 

Z J? . o  ^ 

J .  JL ; 

258.1 

(  3.6) 

237.0 

(  3.5) 

255  6  ( 

4.8) 

275.1 

(  9.2) 

178.2 

(  2.0) 

192.4  ( 

2.1) 

204.9 

(  3.5) 

42.3 

(  1.3) 

44.1  ( 

1.4) 

50.2 

(  2.8] 

106.0 

(  ^-5) 

118.4  ( 

4.9) 

121.5 

(  '.8: 

123.1 

(  3.6) 

135.5  ( 

3.5) 

141.0 

(  5.7] 

150.6 

(  2,4) 

163.3  ( 

2.5) 

172.9 

(  4.9] 

179.6 

(  1.9) 

192.9  ( 

2.3) 

204.8 

(  3.9] 

206.9 

(  3.2) 

223.2  ( 

3.3) 

237.5 

(  6.9] 

231.0 

(  3.8) 

248.5  ( 

5.3) 

269.6 

(  8.3] 

244.6 

V  4.1) 

263.5  ( 

4.0) 

290.4 

(11.8; 

ERIC 
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Table  16-64 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Bridgci  to  1984  Writing  Samples,  by  Grade  and  Subgroup 


Grade  4 

Grade 

8 

Grade 

11 

TOTAL  SAMPLE 

Mean 

1 

X.  »  J  J 

208.2  ( 

0.8) 

220.7  ( 

1.2) 

Standard  Deviation 

1  \^ 

40.1  ( 

0.9) 

39.1  ( 

1.2) 

Percentiles 

5 

101  4  ( 

140.9  ( 

1.5) 

154.6  ( 

2.7) 

10 

2  0"^ 

156.6  ( 

2.8) 

170.7  ( 

2.1) 

25 

181.5  ( 

1.2) 

195.8  ( 

1.8) 

50 

174.4  ( 

2.2) 

208.9  ( 

1.1) 

221.6  ( 

1.4) 

75 

203.5  ( 

:  2.0) 

235.6  ( 

1.1) 

247.3  ( 

2.1) 

228  5  t 

'  2  5^ 

258.7  ( 

1.7) 

269.6  ( 

2.1) 

95 

243  3  i 

273.3  ( 

1.9) 

283.2  ( 

2.6) 

MALE  STUDENTS 

Mean 

164  3  < 

197.9  ( 

1.4) 

211.)  ( 

1.6) 

Standard  Deviation 

42  5  ( 

39.5  ( 

1.2) 

39.1  ( 

1.5) 

Percentiles 

5 

Q3  7  ( 

^  2  5^ 

131.5  ( 

3.5) 

145.0  ( 

3.4) 

10 

109  7  ( 

^  2  5^ 

146.3  ( 

1.8) 

161.3  ( 

4.2) 

25 

135  0  i 

{  1  7^ 

171.7  ( 

2.0) 

186.5  ( 

1.9) 

50 

165.6  1 

[  2.4) 

198.3  ( 

1.7) 

V     1  ( 

1.5) 

75 

194.4  ( 

[  2.6) 

225.1  ( 

2.4) 

?     .7  ( 

2.2) 

90 

218.7  ( 

[  3.5) 

248.3  ( 

2.0) 

260.4  ( 

4.0) 

95 

232.7  ( 

[  4.9) 

261.2  ( 

2.5) 

274.0  ( 

5.3) 

FEMALE  STUDENTS 

Mean 

182.4 

(  1.6) 

218.2  ( 

1.1) 

229.2  ( 

1.4) 

Standard  Deviation 

42.5 

(  1.4) 

38.2  ( 

1.3) 

37.1  ( 

1.2) 

Percentiles 

5 

110.6 

(  4.0) 

154.4  ( 

4.4) 

^67. 8  ( 

3.8) 

10 

127.3 

(  2.4) 

169.5  ( 

3.4) 

1.82.9  ( 

2.8) 

25 

154.1 

(  2.1) 

192.8  ( 

2.6) 

205.2  ( 

2.1) 

50 

183.0 

(  1.8) 

218.6  ( 

2.3) 

229.9  ( 

0.9) 

75 

211.7 

(  2.0) 

244.0  ( 

1.5) 

254.5  ( 

2.9) 

90 

236.6 

(  2.3) 

267.1  ( 

2.5) 

275.6  ( 

2.9) 

95 

251.9 

(  5.6) 

280.0  ( 

3.9) 

288.3  ( 

3.6) 

erJc 


416 


437 


Table  16-64  (continued) 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Bridge  to  1984  Writing  Samples,  by  Grade  and  Subgroup 


Grade  4 


Grade  8 


Grade  11 


WHITE  STUDENTS 

Mean  180.0 

Standard  Deviation  41.8 
Percentiles 

5  110.2 

10  126.0 

25  151.6 

50  181.0 

75  208 . 7 

90  2?.2.6 

95  247.6 

BUCK  STUDENTS 

Mean  150.7 

Standard  Deviation  42.4 
Percentiles 

5  81.1 

10  96.0 

?.5  121.8 

50  15^..  5 

75  179.2 

90  206.2 

95  220.4 

HISPANIC  STUDENTS 

Mean  162.2 

Standard  Deviation  43.1 
Percentiles 

5  93.5 

10  106.9 

25  131.2 

50  161.7 

75  191.5 

90  217.9 

95  234.5 


1.6 
1.1 

2.7 
2.1 
1.7 
2.5 
2.0 
3.4 
4.5 


4.9 


3.6 
1.4 

5.4 
3.9 
4.6 
5.9 
7.4 
5.5 
6.1 


213.1  (  1.0) 
39.5  <  0.9) 


146.4 
162.3 
187.6 
213.7 
240.1 
262.2 
277.2 


190.1 
37.7 

127.2 
141.4 
165.2 
189.9 
215.6 
238  5 
251.7 


197, 
38 


130.6 
147.2 
172.9 
197.9 
223.4 
244.4 
258.8 


1.7) 
2.8) 
1.7) 
^.3) 
1.6) 
2.4) 
2.5) 


2.3) 
1.8) 

3.8) 
5.8) 
2.4) 
2.4) 
2.8) 
3.7) 
6.1) 


2) 
9) 


8.2) 
4.4) 
4  <)) 


6.6) 
3.9) 
5.1) 


225.  i 
37.9 

161.5 
176.8 
200.8 
226 . 2 
250.7 
273.0 
286.0 


206.9 
38.0 

143,2 
158.0 
182.6 
206.7 
232.1 
257.1 
267.7 


202.0 
41.1 

132.7 
148.4 
176.6 
201.8 
229.4 
253.4 
268.1 


ERIC 


417 

438 


Table  16-65 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Main  Focused-BIB  Civics  Samples,  by  Grade  and  Subgroup 


Grade  4 


Grade  8 


Grade  12 


TOTAL  SAMPLE 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 

MALE  STUDENTS 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 

FEMALE  STUDEtlTS 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 


214.0 
8.6 

163.7 
176.5 
195.9 
215.7 
233.3 
248.9 
258.4 


214.8 
29.1 

164.1 
177.5 
196.2 
216.0 
234.0 
250.7 
261.8 


213.3 
28.1 

163.4 
175.6 
195.6 
215.4 
232.8 
247.7 
255.6 


0.9 
0.6 

1.6 
3.8 
1.4 
1.2 
1.3 
2.4 
2.4 


1.3 
0.9 

4.1 
2.9 
2.0 
1.3 
1.9 
3.7 
2.3 


1.1 
0.8 

2.1 
2.4 
1.9 
2.1 
1.4 
1.8 
2.3 


259.7  (0.9 
35.7  (0.6 

198.1  (2.2 

212.8  (1.5 

237.1  (1.3 

261.2  (1.6 
284.2 
304.1 
315.3 


(1.3 
(1.3 
(1.1 


258.7  (1.1 
38.7  (1.0 


190.0  (3.3 
207.6  (2.0 
233.9 
260.7 
286.1 
307.0  (2.4 
318.0  (2.6 


(1.6 
(1.7 
(1.3 


''60.6  (0.9 
32.6  (0.6 

205.2  (3.5 

218.3  (1.2 

239.4  (0.8 

261.6  (0.8 

282.7  (1.4 
301.2  (1.8 
312.6  (3.1 


296.3  (1.1 
37.8  (0.6 


(2.4 
(1.6 


228.4 
247.0 
273.0  (1.3 

299.0  (1.0 

322.1  (1.4 
342.0  (1.2 
353.6  (1.2 


298.6  (1.6 
41.1  (1.0 

221.8  (4.7 

244.0  (3.0 
273.8  (2.4 
302.5  (1.5 
326.8  (2.4 

347.7  (2.7 

360.1  (3.2 


204.1  (1.1 
34.3  (0.8 


234.1 
249.1 
272.3 


(2.4 
(1.7 
(1.6 


296.0  (1.5 


317.9 
336.4 
346.5 


(0.9 
(1.9 
(2.3 
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Table  16-65  (continued) 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Main  Focused-BIB  Civics  Samples,  by  Grade  and  Subgroup 


Grade  4 


Grade  8 


Grade  12 


203.9  (1.0 

221.3  (2.1 
237.8  (1.7 
252.5  (1.9 

261.4  (3.0 


198.1  (2.2 
27.0  (1.4 


WHITE  STUDENTS 

Mean  220.0  (1.0 

Standard  Deviation  26.5  (0.8 
Percentiles 

5  173.  '.9 

10  185.  ,i..4 

25 

50 

75 

90 

95 

BLACK  STUDENTS 
Mean 

Standard  Deviation 

Percentiles 

5  151.2  (4.8 

10  163.2  (2.9 

25  180.5  (4.2 

50  200.0  (1.9 

75  216.8  (3.6 

90  231.1  (1.7 

95  239.2  (5.4 

HISPANIC  STUDENTS 

Mean  199.5  (i.9 

Standard  Deviation  28.8  (1.3 
Percentiles 

5  150.5  (5.4 

10  164.0  (2.9 

25  180.4  (2.2 

50  201.0  (1.9 

75  217.6  (2.0 

90  235.3  (6.1 

95  246.4  (6.3 


266.3 
34.2 

206.4 
222.0 
245.1 
268.4 
289.7 
308.6 
319.7 


243.6 
33.4 

188.0 


.1 
.3 
.2 


202. 
221. 
244. 
267.0 
285.6 
297.4 


240.6 
33.8 

180.9 
196.6 
219.3 
243.0 
263.1 
282.0 
293.0 


1.2 
0.8 

2.2 
i.8 
2.3 
1.4 
1.1 
1.4 
2.8 


1.9 
1.0 

3.9 
4.0 
3.8 
2.7 
2.3 
1.8 
9.7 


1.7 
1.4 

7.3 
4.0 
2.1 
1.4 
1.6 
2.2 
3.4 


301.9 
35.7 

240.8 
255  7 
279.9 
304.3 
326.1 
344.8 
356.1 


273.8 
36.3 

209.4 
225.1 
250.4 
276.2 
298.9 
318.3 
330.0 


279.2 
37.6 

209.4 
226.9 
256.4 
282.8 
305.8 
323.5 
336.3 
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Table  16-66 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Civics  Bridge  Samples,  by  Age  and  Subgroup 


Age  13 


Age  17 


TOTAL  SAMPLE 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 

MALE  STUDENTS 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 

FErLILE  STUDENTS 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 


50.0 
10.0 

34.0 
37.5 
43.3 
49.7 
56.4 
62.9 
67.?. 


50.5 
'-0.4 

33.6 
37.5 
43.5 
50.1 
57.3 
64.2 
68.2 


49.5 
9.6 

34.2 
37.5 
43.1 

55.6 
61.3 
65.4 


0.4 
0.2 


0.4 
0.5 
0.7 

o.r 

1.5 


0.6 
0.3 

0.9 
0.9 
0.6 
0.7 
0.5 
0.9 
1.0 


0.4 
0.3 

0.9 
1.1 
0.4 
0  4 
0.5 
1.0 
1.9 


59.6 
12.3 

39.1 
44.3 
52.0 
59.8 
68.1 
74.8 
79.1 


61.2 
13.0 

39.2 
44.9 
52.8 
61.7 
70.4 
77.3 
81.0 


58.2 
11.5 

39.1 
43.7 
51.4 
58.7 
65.8 
71.9 
75.9 


0.5 
0.4 

1.6 
0.6 
0.6 
0.7 
0.8 
0.6 
0.9 


0.7 
0.5 

2.7 
1.0 
0.8 
1.4 
0.8 
1.3 
1.4 


0.6 
0.5 

2.1 
0  7 
0.8 
0.4 
1.0 
0.8 
1.2 


ERIC 
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Table  i    66  (continued) 


Utighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Civics  Bridge  Samples,  by  Age  and  Subgroup 


Age  13 


Age  17 


WHITE  STUDENTS 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 

BLACK  STUDENTS 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 

HISPANIC  STUDENTS 
Mean 

Standard  Deviation 
Percentiles 
5 

10 

25 

50 

75 

90 

95 


51.2 
9.8 

35.5 
j9.0 
44.6 
50.8 
57.4 
64.1 
68.2 


45.7 
8.6 


31, 
34. 
39, 
45. 
51. 
57.2 
60.4 


45.5 
10.5 

28.8 
32.0 
38.7 
45.3 
51.7 
60.2 
63.4 


0.5 
0.3 

1.0 
0.6 
0.5 
0.5 
0.6 
0.9 
0.8 


0.6 
0.4 

1.0 
1.4 
0.8 
0.7 
0.8 
1.0 
1.1 


1.8 
0.9 

1.7 
2.5 
2.0 
2.3 
2.2 
3.5 
3.4 


61.4 
11.7 

42.2 
47.7 
54.1 
61.6 
69.2 
76.1 
80.0 


53.1 
12.4 

32.2 
38.5 
45.2 
52.4 
61.5 
69.6 
73.6 


53.8 
11.5 

35.0 
39.3 
46.1 
53.8 
62.1 
68.2 
71.4 


0.6 
0.4 

1.2 
1.5 
0.8 
1.0 
0.6 
0.7 
1.0 


1.0 
0.6 

4.6 
1.5 
0.7 
1.0 
1.4 
1.3 
1.3 


1.7 
1.0 

5.5 

6.4 

3.2 
3.5 
1.3 
2.2 
(5.3 


ERIC 
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Table  16-67 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Main  U.S.  History  Samples,  by  Grade  and  Subgroup 

Grade  4  Grade  8  Grade  12 

TOTAL  SAMPLE 

Mean  220.6        ))  263.9  (0.7)  295.0  (1.0) 

Standard  Deviation  30.0  (0.5)  34.1  (0.4)  36.1  (0.6) 
Percentiles 

5  169.7  (1.2)  204.3  (2.4)  231.8  (1.0) 

10  181.0  (1.2)  219.0  (1.1)  246.7  (2.3) 

25  201.3  (1.1)  240.9  (1.1)  270.4  (1.2) 

50  221.5  (1.0)  265.0  (0.6)  295.0  (1.1) 

75  240.3  (0.9)  288.8  (0.6)  318.9  (0.8 

90  258.7  (1.7)  306.6  (1.2)  341.5  (0.8) 

95  268.9  (1.7)  317.2  (0.7)  348.6  (1.4) 

MALE  STUDENTS 

Mean  222.9  (1.2)  266.2  (1.0)  ?98.5  (1.3) 

Standard  Deviation  30.9  (0.7)  36.0  (0.7)  38.6  (0.8) 
Percentiles 

5  170.3  (1.5)  202.5  (2.7)  229.2  (3.1) 

10  182.0  (2.0)  218.2  (2.1)  244.7  (2.1) 

25  202.6  (1.4)  242.1  (1.3)  273.3  (2.6) 

50  225.1  (1.8)  267.3  (0.9)  302.0  (4.3) 

75  242.7  (1.4)  291.4  (0.8)  324.4  (2.5) 

90  262.4  (1.6)  311.6  (1.6)  344.6  (0.8) 

95  272.1  (1.8)  320.1  (1.1)  353.9  (2.3) 

FEMALE  STUDENTS 

Mean  218.2  (1.0)  261.6  (0.8)  291.8  (1.1) 

Standard  Deviation  23.8  (0.6)  31.9  (0.5)  33.4  (0.6) 
Percentiles 

5  169.0  (1.8)  206.2  v3.1)  234.1  (1.8) 

10  180.1  (1.2)  219.8  (1.9)  248.3  (2.0) 

25  199.7  (1.7)  240.1  (1.0)  268.7  (0.9) 

50  218.6  (1.2)  263.3  (0.5)  291.8  (0.8) 

75  238.2  (1.3)  285.7  (1.2)  316.2  (0.6) 

90  253.7  (1.9)  299.8  (1.0)  335.8  (?.2) 

95  264.6  (2.6)  313.4  (1.5)  344.6  (0./) 


4^3 
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Table  16-67  (continued) 


Weighued  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Main  U.S.  History  Samples,  by  Grade  and  Subgroup 


Grade  4 


Grade  8 


Grade  12 


WHITE  STUDENTS 


Mean 

227. 

5 

(1. 

0) 

270. 

4 

(0.8) 

301. 

1 

(1.2) 

Standard  Deviation 

27. 

0 

(0. 

6) 

32. 

2 

(0.6) 

34. 

1 

(0.6) 

Percentiles 

5 

181. 

8 

(2. 

3) 

212. 

9 

(1.5) 

240. 

5 

(4.7) 

10 

192. 

5 

(1. 

9) 

229. 

5 

(1.4) 

255. 

8 

(1.7) 

25 

209. 

5 

(1. 

1) 

251. 

4 

(2.0) 

281. 

5 

(2.0) 

50 

229. 

2 

(1. 

0) 

270. 

8 

(1.2) 

304. 

0 

(1.6) 

75 

244. 

4 

(1. 

0) 

291. 

8 

(0.6) 

322. 

3 

(1.4) 

90 

262. 

6 

(1. 

7) 

311. 

2 

(1.0) 

343 . 

5 

(l.i) 

95 

271. 

8 

(1. 

8) 

319. 

3 

(0.9) 

350. 

9 

(1.4) 

BLACK  STUDENTS 

Mean 

199. 

5 

(1. 

9) 

246 

0 

(1.5) 

274. 

4 

(1.7) 

Standard  Deviation 

27 

7 

(1. 

1) 

30 

4 

(1.0) 

33 

7 

(0.9) 

Percentiles 

5 

154 

5 

(5 

1) 

195 

2 

(3.9) 

218 

0 

(4.5) 

10 

163 

4 

(4 

7) 

207 

5 

(3.0) 

230 

9 

(2.7) 

25 

180 

5 

(1 

3) 

227 

.  5 

(2.1) 

252 

1 

(2.2) 

50 

199 

.5 

(3 

0) 

245 

.  1 

(1.6) 

2.1 5 

r 
.  0 

(3.4) 

75 

217 

.5 

(1 

2) 

265 

.3 

(1.9) 

296 

.3 

(2.:)) 

90 

236 

.9 

(3 

.2) 

286 

.7 

(1.6) 

317 

.6 

(0.9) 

95 

244 

.9 

(2 

.8) 

295 

.0 

(2.4) 

327 

.8 

(4.6) 

HISPANIC  STUDENTS 

Mean 

202 

.7 

(1 

.7) 

244 

.3 

(1.9) 

273 

.9 

(1.8) 

Standard  Deviation 

28 

.7 

(0 

.9) 

33 

.7 

(0.9) 

34 

.6 

(1.2) 

Percentiles 

5 

153 

.4 

(5 

.9) 

187 

.7 

(5.9) 

217 

.  i 

(4.4) 

10 

165 

.7 

(2 

.8) 

202 

.1 

(3.2) 

229 

.7 

(2.9) 

25 

182 

.7 

(1 

.3) 

221 

.8 

(2.2) 

252 

.3 

(3.0) 

50 

204 

.1 

(2 

.5) 

244 

.4 

(2.5) 

212 

.7 

(3.2) 

75 

222 

.5 

(1 

.6) 

266 

.7 

(1.3) 

295 

.8 

(2.9) 

90 

239 

.5 

(0 

.9) 

288 

.5 

(2.0) 

318 

.6 

(1.0) 

95 

247 

.3 

(4 

.0) 

296 

.7 

(1.9) 

329 

.1 

(7.9) 

ERIC 


423 


Table  16-68 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Main  Geography  Sample,  by  Grade  and  Subgroup 


Grade  12 

TOTAL  SAMPLE 

Mean  293.1  (1.0) 

Standard  Deviation  37.0  (0.8) 
Percentiles 

5  227.5  (2.7] 

10  242.2  (1.6) 

25  268.7  (2.0) 

50  295.5  (1.5) 

75  319.1  (0.9] 

90  338.6  (1.3) 

95  349.1  (3.3] 

MALE  STUDENTS 

Mean  301.2  (1.6] 

Standard  Deviation  38.3  (1.0) 
Percentiles 

5  231.6  (2.2] 

10  247.1  (2.8) 

25  275.6  (2.8] 

50  305.6  (2.0) 

75  328.4  (1.6) 

90  346.6  (2.3) 

95  358.3  (4.8) 

FEMALE  STUDENTS 

Mean  285.7  (1.2) 

Standard  Deviation  34.2  (0.9) 
Percentiles 

5  224.3  (2.5) 

10  238.9  (2.2) 

25  263.7  (l.j) 

50  289.2  (1.2) 

75  310.0  (1.2) 

90  327.6  (2.2) 

95  337.9  (3.6] 
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Table  16-68  (continued) 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Main  Geographj  Samples,  by  Grade  and  Subgroup 


Grade  12 

WHITE  STUDENTS 

Mean  301.1  (1.1) 

Standard  Deviation  33.7  (0.9) 
Percentiles 

5  242.3  (2.8) 

10  256.4  (2.9) 

25  279.7  (1.4) 

50  302.7  (1.4) 

75  32."^. 7  (1.4) 

90  341.9  (2.1) 

95  352.9  (2.1) 

BLACK  STUDENTS 

Mean  258.4  (2.0) 

Standard  Deviation  32.0  (1.2) 
Percentiles 

5  207.3  (3.7) 

10  216.8  (3.5) 

25  234.7  (2.9) 

50  259.4  (3.5) 

75  ?e2.2  (3.8) 

90  298.8  (2.5) 

95  310.5  (7.8) 

HISPANIC  STUDENTS 

Mean  271.8  (3.9) 

Standard  Deviation  35.0  (1.6) 
Percentiles 

5  214.4  (8.6) 

10  225.0  (7.4) 

25  245.8  (3.4) 

50  273.1  (5.6) 

75  296.9  (2.8) 

90  317.8  (8.7) 

95  331.1  (3.0) 
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Table  16-69 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Bridge  to  1986  Mathematics  Samples,  by  Age  and  .  abgroup 


Age  9  Age  13  Akie  17 

TOTAL  SAMPLE 
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324 

5  ( 

:  2 

2) 

90 

269, 

1  ( 

:  1 

5) 

311 

5  ( 

:  1.4) 

341 

9  ( 

:  1 

8) 

95 

278, 

4  ( 

:  2 

4) 

322 

8  ( 

:  1.8) 

350 

8  ( 

:  3 

1) 

426 

4d7 


Table  16-69  (continued) 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

wich  Standard  Errors 
for  Bridge  to  1986  Mathematics  Samples,  by  Age  and  Subgroup 


Age  9 


Age  13 


Age  17 


WHITE  STUDENTS 


Mean 

234.5  ( 

:  1. 

2) 

279. 

1  ( 

0.9) 

309. 

5 

(  1. 

4) 

Standard  Deviation 

30.4  ( 

:  0. 

7) 

29. 

4  ( 

0.4) 

28. 

9 

(  0. 

8) 

Percentiles 

5 

182.4  ( 

:  3. 

3) 

231. 

6  ' 

1.5) 

259. 

3 

(  1. 

7) 

10 

194.4  ( 

:  1. 

8) 

241. 

9  ( 

1.4) 

271. 

0 

(  1. 

2) 

25 

214.4  ( 

:  2. 

0) 

259. 

1  ( 

1.3) 

290. 

2 

(  2. 

7) 

50 

o  o  c    n  i 

235.9  1 

I     J-  . 

3) 

o  "7  n 
2/9  . 

U  ( 

1.4) 

JIU . 

I 

j) 

75 

255.6  ( 

;  1 

7) 

299 

1  ( 

1.5) 

^29 . 

3 

(  2. 

7) 

90 

272.6  ( 

;  1 

4) 

317 

3  ( 

1.3) 

346. 

6 

(  2 

0) 

95 

282.4  ( 

;  1 

3) 

327 

8  ( 

:  1.4) 

356 

(  2. 

9) 

BLACK  STUDENTS 

Mean 

206.3 

[  2 

6) 

250 

3  ( 

:  1.2) 

289 

2 

(  2 

1) 

Standard  Deviation 

32.7 

[  1 

3) 

27 

9  ( 

:  1.0) 

27 

1 

(  1 

3) 

Perc.ntiles 

5 

152.6 

[  2 

2) 

203 

7  ( 

:  1.7) 

243 

9 

(  5 

9) 

10 

164.3 

[  ^ 

0) 

214 

.5  ( 

:  1.8) 

254 

5 

(  3 

4) 

25 

183.8 

[  ^ 

8) 

231 

9  ( 

:  2.5) 

270 

3 

(  1 

.6) 

50 

206.6 

(  3 

6) 

250 

.9  ( 

;  2.3) 

288 

6 

(  1 

.3) 

75 

229.2 

(  2 

1) 

i!69 

.5  ( 

:  1.4) 

307 

4 

(  3 

4) 

90 

247.7 

(  2 

2) 

284 

.8  1 

;  2.0) 

325 

.1 

(  3 

6) 

95 

258.2 

(  2 

.0) 

294 

.1  ( 

:  2.1) 

334 

.1 

(  6 

.4) 

HISPANIC  STUDENTS 

Mean 

215.9 

(  3 

.4) 

254 

.7  ( 

;  3.9) 

294 

(  3 

.5) 

Standard  Daviation 

33.1 

(  1 

.7) 

30 

.3 

[  2.2) 

27 

.6 

(  2 

.6) 

Percentiles 

5 

162.1 

(  6 

.8) 

201 

.1 

[  7.6) 

2^.9 

.1 

(  9 

.7) 

10 

173.7 

(  7 

.0) 

214 

9 

[  4.3) 

259 

.3 

(11 

.4) 

25 

193.3 

(  4 

.0) 

234 

.4 

[  5.4) 

27k 

.9 

(  6 

.9) 

50 

216.3 

(  5 

.4) 

257 

.2 

[  4.6) 

292 

.8 

(  ^ 

.8) 

75 

239.1 

(  5 

.4) 

274 

.2 

I  3.0) 

313 

.9 

(  2 

.9) 

90 

259.8 

(  ^ 

.7) 

289 

.1 

(  5.0) 

330 

.1 

(  2 

.9) 

95 

269.2 

(  9 

.7) 

299 

.4 

(  3.4) 

341 

.9 

(  9 

.3) 

ERIC 
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Table  16-70 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Bridge  to  1986  Science  Samples,  by  Age  and  Subgroup 


Age  9 


Age  13 


Age  17 


TOTAL  SAMPLE 

Mean  228.9 

Standard  Deviation  41.0 
Percentiles 

5  160.5 

10  175.8 

25  201.3 

50  230.0 

75  257.0 

90  280.7 

95  293.9 

MALE  STUDENTS 

Mean  232.1 

Standard  Deviation  41.2 
Percentiles 

5  163.5 

10  178.7 

25  204.6 

50  233.3 

75  259.8 

90  284.0 

95  298.3 

FEMALE  STUDENTS 

Mean  225.7 

Standard  Deviation  40.5 
Percentiles 

5  157.5 

10  173.2 

25  198.7 

50  227.0 

75  253.9 

90  276.3 

95  289.4 


1.3 
0.7 

1.9 
1.5 


1.6 
1.0 

2.8 
2.0 
1.8 
1.9 
2.5 
2.6 
4.5 


1.6 
0.8 

1.9 
3.1 
1.5 


257.3 
37.2 

194.4 
208.8 
232.1 
258.1 
282.9 
304.3 
317.7 


262.2 
37.5 

198.6 
213.1 
237.0 
262.9 
287.7 
309.8 
322.9 


252.4 
36.3 

190.8 
205.2 
228.3 
253.5 
277.4 
297.7 
310.2 


0.9 
O.i 

2.4 
1.1 
1.3 
1.0 
1.3 
1.4 
2.4 


1.2 
0.8 

2.2 
1.9 
1.6 
1.2 
1.7 
2.3 
3.0 


1.0 
0.9 

2.9 
2.6 
2.1 
0.8 
1.3 
2.1 
2.8 


294.2 
41.4 

225.1 
240.9 
266.5 
294.5 
322.2 
347.7 
362.7 


302.5 
41.6 

232.2 
249.7 
275.4 
302.7 
331.1 
357.3 
370.5 


235.6 
39.4 

220.0 
235.1 
258.9 
287.0 
312.7 
3j5.3 
349.1 


428 

4d9 


Table  16-70  (continued) 


Weighted  Proficiency  Means,  Standard  Deviations,  and  Percentiles 

with  Standard  Errors 
for  Bridge  to  1986  Science  Samples,  by  Age  and  Subgroup 


Age  9  Age  1j  Age  17 


WHITE  STUDENTS 


iriean 

237. 

4 

( 

1. 

3) 

265. 

2 

( 

0. 

9) 

301. 

9 

( 

1. 

7) 

Standard  Deviation 

37. 

4 

( 

0. 

7) 

34 

0 

( 

0. 

7) 

38. 

0 

( 

1 

5) 

Percentiles 

5 

176 

0 

( 

1 

6) 

209 

1 

( 

1 

1) 

239 

1 

( 

4 

5) 

10 

189 

1 

( 

1 

1) 

221 

9 

( 

1 

8) 

254 

3 

( 

5 

0) 

25 

211 

8 

( 

1 

.4) 

242 

3 

( 

1 

1) 

276 

.4 

( 

3 

.0) 

50 

238 

.3 

( 

1 

.6) 

265 

3 

( 

0 

.9) 

301 

.9 

( 

2 

.0) 

75 

262 

.7 

( 

1 

.3) 

287 

.9 

( 

1 

.5) 

327 

r 

( 

2 

.4) 

90 

285 

.2 

( 

1 

.8) 

308 

.6 

( 

2 

.0) 

351 

.3 

( 

3 

.1) 

95 

298 

.3 

( 

4 

.2) 

320 

.9 

( 

1 

.9) 

364 

.5 

( 

2 

.6) 

BLACK  STUDENTS 


Mean 

200. 

1 

( 

2. 

5) 

229 

4 

( 

1. 

2) 

260. 

0 

(  3.4) 

Standard  Deviation 

38. 

7 

( 

1 

5) 

33 

5 

( 

1 

2) 

38 

6 

(  2.7) 

Percentiles 

5 

135 

7 

( 

3 

0) 

175 

0 

( 

3 

3) 

200 

0 

J. 

(11.1) 

10 

149 

3 

( 

5 

4) 

186 

7 

( 

4 

1) 

212 

3 

(  8.2) 

25 

173 

5 

( 

3 

8) 

206 

8 

( 

1 

9) 

236 

.0 

(  3.2) 

50 

200 

2 

( 

2 

.6) 

228 

.6 

( 

3 

0) 

259 

.0 

(  2.8) 

75 

226 

.3 

( 

2 

.9) 

252 

.7 

( 

2 

.6) 

283 

.9 

(  5.4) 

90 

251 

.3 

( 

4 

.5) 

272 

.7 

( 

2 

.5) 

309 

.2 

(  4.4) 

95 

263 

.8 

( 

4 

.1) 

283 

.6 

( 

4 

.2) 

323 

.6 

(  9.6) 

HISPANIC  STUDENTS 


Mean 

201 

0 

( 

6.1) 

229. 

3 

( 

4. 

2) 

281. 

8 

(  5.2) 

Standard  Deviation 

42 

0 

( 

2.8) 

35. 

5 

( 

2. 

1) 

37. 

3 

(  2.8) 

Percentiles 

(  6.8) 

5 

132 

3 

( 

6.4) 

172 

1 

( 

8 

9) 

225 

8 

10 

146 

8 

( 

9.1) 

181 

9 

( 

5 

9) 

236 

6 

(  4.9) 

25 

172 

0 

( 

8.6) 

202 

0 

( 

b 

8) 

254 

7 

(  6.7) 

50 

201 

.0 

( 

8.6) 

229 

8 

( 

4 

.0) 

282 

6 

(11.6) 

75 

228 

.7 

( 

6.5) 

254 

.7 

( 

5 

.5) 

303 

.2 

(  9.3) 

90 

255 

.6 

( 

5.4) 

276 

.9 

( 

4 

6) 

334 

.1 

(13.8) 

95 

270 

.2 

( 

4.8) 

289 

.2 

( 

8 

.4) 

350 

.6 

(  7.5) 

429 

450 


Table  16-71 


Weighted  Response  Percentages  and  Reading  Proficiency  Means  with  Standard  Errors 
Grade  4,  by  Gender,  for  Main  Focused- BIB  Reading  Samples 


GEHDER  OF  SUBJECT    (NAEP  ID:  SEX) 


ri 

MALE 

FEMALE 

TOTAL 

HJO  X 

m.A  ) 

51. 7( 

1.1) 

46. 3( 

1.1) 

0.0 

226. 6( 

1.5) 

234. 4( 

1.2) 

GEKDER 

HALE 

234S 

445,04e( 

3X) 

100. 0( 

0.0) 

0.0( 

0.0) 

O.l 

226. 6( 

1.5) 

FEMALE 

2166 

A15.505( 

3X) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0 

234. 4( 

1.2) 

RACE/ETHNICITY 

WHITK 

2557 

599.759( 

2X) 

51. 5( 

1.6) 

46. t( 

1.6) 

0.0 

234. ec 

1.6) 

241. 6( 

1.4) 

BLACK 

654 

137,16A( 

3X) 

50. 7( 

1.9> 

49. 3( 

1.9) 

0.1 

206. 6( 

2.6) 

214. 6( 

2.4) 

HISPANIC 

660 

6a,350( 

3X) 

53. 5( 

2.0) 

46. 5( 

2.0) 

0.0 

202. 7( 

3.4) 

216. 2( 

2.1) 

OTHER 

251 

35,2ei( 

7X) 

54. 6( 

3.4) 

45. 4( 

3.4) 

0.3 

226. 4( 

5.4) 

229. 0( 

4.4) 

FAt<£}iTAL  EDUCATION 


LESS  THAN  H.S. 

226 

36,671(10X) 

57. 2( 

3.2) 

42. 6( 

3.2) 

0.0 

196. 9( 

4.6) 

214. 4( 

3.6) 

GRADUATED  H.S. 

67' 

132.632(  5X) 

50. 3( 

2.7) 

49. 7( 

2.7) 

0.0 

222. 9( 

2.7) 

226. 9( 

2.6) 

SOME  EDUC  ATTET  H.S. 

356 

67,315(  6X) 

49. 4( 

3.6) 

5C  6( 

3.6) 

0.0 

232. 3( 

3.0) 

249  < 

3.1) 

GRADUATED  COLLEGE 

1623 

327,691(  4X) 

53. 4( 

1.4) 

46. t( 

1.4) 

0.0 

236. 5( 

2.1) 

243  7( 

1.6) 

UNiaiOWH 

1626 

292.006(  3X) 

50. 1( 

1.5) 

49. 9( 

1.5) 

0.1 

217. 7( 

1.7) 

226. 2( 

1.6) 

*  CV  is  th«  coo££lql«nt  o£  variation  for  tht  sum  o£  tha  weights. 


ERIC 


4151 

430 


Table  16-72 


Weighted  Response  Percentages  and  Reading  Proficiency  Means  with  Standard  Errors 
Grade  4,  by  Derived  Racc/ethnicity ,  for  Main  Focused-BIB  Reading  Samples 


DERIVED  RACE/EXEHICIXY    (HASP  ID:  DRACE) 


WEICTTED  N  (CV)* 


WHITE 


BLACK 


—    TOTAL  — 

4S34 

860«914( 

IX) 

69 

7( 

0 

.6) 

15 

9( 

0 

5) 

238 

K 

1 

2> 

210 

6( 

I 

9) 

MAT  F 

"yxL  7 

^OH 1 

445,278( 

3X) 

A  / 
*»  I 

i 

15 

6( 

0 

7) 

234 

8( 

1 

8) 

206 

6( 

2 

8) 

FQ1ALE 

91  A7 

415.536( 

3X) 

n  / 

i 

Z) 

1  c 

lb 

0 

9) 

6( 

1 

214 

6( 

2 

A) 

RACE  /ETHN ICT  Tv 

599.890( 

2X) 

100 

n  / 

0 

0) 

0 

0( 

0 

0) 

1  / 

I-} 

BLACK 

855 

137.2g6( 

32) 

Q 

Q 

r\t 
tP  V 

n 
u 

210 

6( 

1 

9) 

HISPANIC 

869 

88, 350 ( 

3X) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

OTHER 

252 

35,378( 

7X) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

PARENTAL  EDUCATION 

LESS  THAK  H.S. 

228 

38.6>1(10Z) 

65 

8( 

3 

9) 

15 

K 

2 

5) 

210 

0( 

3) 

187 

2( 

> 

A) 

GRADUATED  H.S. 

675 

1?2,632( 

5X) 

73 

7( 

1 

7) 

H 

6( 

1 

5) 

231 

0( 

2 

6) 

211 

3( 

3 

5) 

SOME  EDUC  AFTER  H.S. 

358 

67,315( 

6X) 

7A 

0( 

2 

1) 

13 

7( 

1 

9) 

2A8 

0( 

2 

3) 

219 

9( 

6 

0) 

GRADUATED  COLLEGE 

162A 

327,82A( 

•iX) 

72 

0( 

1 

3) 

16 

0( 

1 

0) 

2A8 

3( 

1 

7) 

217 

9( 

3 

1) 

UmCNOWN 

1630 

Z92,23Ar 

3X) 

65 

3( 

1 

A) 

16 

6( 

1 

1) 

230 

3( 

1 

5) 

203 

9( 

3 

0) 

HISPANIC 

10. 3(  0.3) 
209. 9(  2. A) 


10. 6(  0.6) 

202. 7(  3. A) 

9.9(  0.5) 

218. 2(  2.1) 


0.0(  0.0) 

O.OC  0.0) 

100. 0(  0.0) 

209. 9(  2. A) 

0.0(  0.0) 


16. 7( 
197. 6( 

9.7( 
210. 6( 

9.0( 
223. 5( 

7.5( 
220. 0( 
13. 0( 
203. A( 


2.9) 
6.2) 
0.9) 
5.1) 
1.2) 
5.7) 
0.6) 
3.8) 
0.7) 
2.9) 


ASIAN 

2.3(  0.2) 
23A.5(  5.9) 


AMER  IND  UNCLASSIFIED 


O.OC  0.0) 

O.OC  0.0) 

O.OC  0.0) 

56. 8C  A.l) 

234. 5C  5.9) 


O.AC  0.4) 
20A.5C****) 

1.0'  0.3) 
239.6C18.2) 

1.9C  0.7) 
216.6C20.A) 

2. AC  O.A) 
2A7.8C  A. 9) 

3.2C  0.5) 
225. 6C  8.7) 


1.7C  C.2) 
218. 6C  5.1) 


O.OC  0.0) 
213.0C17.0) 


O.OC  0.0) 

O.OC  0.0) 

O.OC  0.0) 

A2.1C  A. 2) 

218. 6C  5.1) 


2.0C  0.7) 
20A.5C  7.6) 

0.9C  0.3) 
19A.7C16.2) 

1.4C  0.6) 
209.5C26.6) 

2.0C  0.3) 
230. 8C  6.6) 

1.9C  O.A) 
213. 2C  7.7) 


O.OC  0.0) 

O.OC  0.0) 

O.OC  0.0) 

I.IC  0.7) 
213.0C17.0) 

O.OC  0.0) 

O.OC  0.0) 

O.OC  0.0) 

O.IC  0.1) 
213.0C17.0) 
O.OC  0.0) 


MISSING 
0.0 


2 

5C 

0 

3) 

1 

8C 

0 

3) 

0 

IC  0 

.0) 

0.0 

232 

oc 

7 

9) 

219 

5C 

6 

7) 

193 

0C37 

5) 

2 

IC 

0 

3) 

1 

7C 

0 

3) 

0 

OC  0 

0) 

0.0 

237 

7C 

6 

3) 

217 

6C 

6 

9) 

251 

IC  9 

5) 

0.0 
0.0 
0.0 
0.0 

0.0 
0.0 
0.0 
0.0 
0.0 


*  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 
C****)  Standard  error  is  greater  than  99.9. 


ERIC  452 


431 


4:->  .1 


Table  16-73 

Weighted  Response  Percentages  and  Reading  Proficiency  Means  with  Standard  Errors 
Grade  4,  by  Parental  Education,  for  Main  Focused-BIB  Reading  Samples 


PAREHTS'  EDUCATION    (NAEP  ID:  PARED) 


-  TOTAL 


GENDER 
MALE 

FEMALE 


RACE/ETHKICITY 
WHITE 

BLACK 

HISPANIC 

OTHER 

PARENTAL  \DUCATION 
LESS  THAN  H.S. 

GRADUATED  H.S. 

SCXffi  E^UC  AFTER  H.S. 

GRADUATED  COLLEGE 

umarowN 


N 

WEIGHTED  K 

(CV)* 

NOT  HS 

GRAD  HS 

POST 

HS 

GRAD 

COL 

UNKNOWN 

MISSING 

4515 

858,677( 

IX) 

4 

5( 

0.4) 

15 

4( 

0 

8) 

7 

8( 

0 

5) 

38 

2( 

1 

5) 

34 

0( 

1. 

1) 

0 

3 

204 

4( 

3.2) 

225 

9( 

2 

1) 

240 

8( 

2 

1) 

240 

9( 

1 

3) 

221 

9( 

1. 

4) 

'^333 

443,602( 

3X) 

5.0( 

0.6) 

15 

0( 

1. 

0) 

7 

5( 

0 

7) 

39 

5( 

1 

6) 

33. 0( 

1. 

3) 

0 

4 

196 

9( 

4.8) 

222 

9( 

2 

7) 

232 

3( 

3 

0) 

238 

4( 

2 

1) 

217. 7( 

1, 

7) 

2162 

415,075( 

3X) 

4 

0( 

0.5) 

15 

9( 

1 

1) 

8 

2( 

0 

7> 

36 

8( 

1 

8) 

35 

1( 

1. 

4) 

214 

4( 

3.8) 

228 

9( 

2.e) 

249 

2( 

3 

1) 

243. 7( 

1 

6) 

226. 2( 

1 

6) 

2557 

599,750( 

2X) 

4 

2( 

0.6) 

16 

3( 

1 

0) 

8 

3( 

0 

6) 

39 

3( 

1 

9) 

31 

8( 

1. 

4) 

0 

0 

210 

0( 

4.3) 

231. 0( 

2 

6) 

248 

0( 

2 

3) 

248 

3( 

1 

7) 

230 

3( 

1. 

5) 

642 

135,579( 

3X) 

4 

3( 

0.8) 

14 

3( 

1 

6) 

6 

8( 

1 

0) 

38 

8( 

1 

9) 

35 

8( 

1.9) 

1 

3 

107 

2( 

7.4) 

211. 3( 

3 

5) 

219. 9( 

6 

0) 

217 

g( 

3 

1) 

203 

9( 

3. 

0) 

664 

B7,970( 

3X) 

7 

3( 

1.2) 

14 

7( 

1 

2) 

6.9( 

1 

0) 

27 

8( 

2 

2) 

43 

3( 

2. 

1) 

0 

4 

197 

6( 

6.2) 

210. 6( 

5 

1) 

223 

5( 

5 

7) 

220 

0( 

3 

8) 

203 

4( 

2. 

9) 

252 

35,37B( 

7X) 

2 

6( 

0.8) 

7 

2( 

1 

6) 

6 

3( 

1 

6) 

41 

7( 

4 

4) 

42 

2( 

4 

0) 

0 

0 

204. 5( 

9.3) 

218 

1(12 

0) 

213 

7(14 

9) 

239 

.5( 

3 

9) 

221 

0( 

5. 

9) 

226 

3B,G71(10X) 

100 

0( 

0  0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0. 

0) 

0 

0 

204 

4( 

3.2) 

675 

132.632( 

5X) 

0 

0( 

0.0) 

100 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0 

225 

9( 

2 

1) 

356 

67,315( 

6X) 

C 

0( 

0.0'j 

0 

0( 

0 

0) 

100 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0 

240. 8( 

2 

1) 

1624 

327, 824 ( 

4X) 

0 

0( 

0.0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

100 

0( 

0 

0) 

0 

0( 

0.0) 

0 

0 

240 

9( 

1 

3) 

1630 

292. 234 ( 

3X) 

0 

0( 

0.0) 

0.0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

100 

0( 

0 

0) 

0 

0 

221. 9(  1.4) 


*  (TV  is  tho  coefficient  of  variation  for  ths  sum  of  the  woights. 
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Table  16-74 


Weighted  Response  Percentages  and  Reading  Proficiency  Means  with  Standard  Errors 
Grade  8,  by  Gender,  for  Main  Focused-BIB  Roading  Samples 


GENDEP.  OF  SUBJECT    (NAEP  XD:  SEX) 

n         WEIGHTED  N  (CV)*         MALE  FEMALE  MISSING 


—   TOTAL  — 

491,575( 

IX) 

51. 5( 

1.1) 

48. 5( 

1 

1) 

0.0 

256. 4 ( 

1.3) 

269. 6( 

1 

1) 

GEHDER 

MAI.E 

2242 

253,319( 

3X) 

100. 0( 

0.0) 

0.0( 

0 

0) 

0.0 

256. A( 

1.3) 

FEMALE 

2162 

238,257( 

2X) 

0.0( 

0.0) 

100. 0( 

0 

0) 

0.0 

269. 6( 

1 

1) 

RACE/ETHNICITY 

WHITE 

2694 

3A3,252( 

2X) 

51. 5( 

1.2) 

48. 5( 

1 

2) 

0.0 

262. 8( 

1.8) 

276. 1( 

1 

4) 

BLACK 

772 

7A,650( 

5X) 

49. 3( 

3.3) 

50. 7( 

3 

3) 

0.0 

238. 9( 

2.5) 

252. 4 ( 

2 

2) 

HISPANIC 

711 

52,A13( 

AX) 

54. 8( 

2.3) 

4b. ?( 

2 

3) 

0.0 

237. 7( 

2.3) 

250. 7( 

2. 

7) 

OTHER 

227 

21,260( 

8X) 

51. 8( 

3.2) 

48. 2( 

3. 

2) 

0.0 

260. 7( 

4.4) 

271. 9( 

3. 

3) 

PARENTAL  EDUCATION 


LESS  THAN  H.S. 

375 

36,427( 

8X) 

40. 4( 

2.9) 

59. 6( 

2.9) 

0 

0 

236. 8( 

3.9) 

250. 2( 

2.2) 

GRADUATED  H.S. 

1193 

136,783( 

3X) 

51. 7( 

1.6) 

48. 3( 

1.6) 

0 

0 

249. 9( 

2.0) 

263. 2( 

1.7) 

SOME  EDUC  AFTER  H.S. 

731 

89.177( 

4X) 

47. 3( 

1.6) 

52. 7( 

1.6) 

0 

0 

261. 2( 

2.1) 

z75.3( 

1.8) 

GRADUATED  COLLEGE 

1  39 

189.833( 

4X) 

53. 8( 

1.6) 

46. 2( 

1.6) 

0 

0 

266. 4( 

1.8) 

280. 2( 

1.5) 

UHiaiOWN 

403 

38,033( 

7Z) 

59. 6( 

2.8) 

40. 4( 

2.8) 

0 

0 

237. 6( 

2.7) 

248. 6( 

2.3) 

*  CV  is  the  cosffici«nt  of  variation  ior  the  sura  of  the  weights. 
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Table  16-75 


Weighted  Response  Percentages  and  Reading  Proficiency  Means  with  Standard  Errors 
Grade  8,  by  Derived  Race/ethnicity,  for  Main  Focused-BIB  Reading  Samples 


DERIVED  RACE/ETHNICITY    (NAEP  ID:  DRACE) 


K 


WEIGHTED  N  (CV)* 


WHITE 


BUCK 


HISPANIC 


ASIAN 


AMER  IND  UNCLASSIFIED 


MISSING 


—    TOTAL  ~ 

4404 

491.575( 

IX) 

69 

8( 

0. 

7) 

15, 

2( 

0. 

7) 

10 

7( 

0 

4) 

3 

0( 

0 

3) 

1 

3( 

0. 

2) 

0 

K 

0 

0) 

0 

0 

269 

2( 

1. 

3) 

245 

7( 

2 

0) 

243 

6( 

2 

1) 

274 

y( 

2 

7) 

248 

K 

5. 

5) 

206 

1(26 

7) 

GENDER 

K 

1) 

MALE 

2242 

253,3ig( 

3X) 

69 

8( 

1. 

0) 

14 

5( 

0 

8) 

11 

3( 

0 

6) 

3 

0( 

0 

4) 

1 

3( 

0 

3) 

0 

0 

0 

0 

262 

8( 

1 

8) 

238 

9( 

2 

5) 

237 

7( 

2 

3) 

273 

0( 

3 

1) 

236 

7( 

7 

8) 

206 

1(26 

7) 

FEMALE 

2162 

238.257( 

2X) 

69 

9( 

1 

3) 

15 

9( 

1 

3) 

10 

0( 

0 

7) 

3 

0( 

0 

4) 

1 

3( 

0. 

3) 

C 

0( 

0 

0) 

0 

0 

276 

K 

1 

4) 

252 

4( 

I 

2) 

250 

7( 

2 

7) 

277 

0( 

4 

5) 

260 

2( 

5 

3) 

RACE/ETHNICITY 

0( 

0) 

WHITE 

2694 

343,252( 

2X) 

100 

.0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0? 

0 

0( 

0 

0) 

0 

0 

0 

0 

269 

2( 

1 

3) 

BUCK 

772 

74. 650 ( 

5X) 

0 

0( 

0 

0) 

100 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0 

245 

7( 

2 

.0) 

HISPANIC 

711 

52.i(13( 

4X) 

0 

.0( 

0 

0) 

0 

0( 

0 

0) 

100 

0( 

0 

0) 

0 

.0( 

0 

0) 

0 

0( 

0 

0) 

0 

.0( 

0 

0) 

0 

0 

243 

6( 

2 

1) 

OTHER 

227 

21,260( 

8X) 

0 

.0( 

0 

0) 

0 

0' 

0 

0) 

0 

.0( 

0 

0) 

68 

.9( 

4 

9) 

2r 

9( 

4 

9) 

1 

2( 

0 

7) 

0 

0 

274 

.9( 

2 

7) 

2i8 

K 

5 

5) 

206 

1(26 

7) 

PARENTAL  EDUCATION 


LESS  THAN  H.S. 

375 

36,427( 

8X) 

51. 9( 

3.5> 

16 

8( 

2.9) 

27. 2( 

2. 

1) 

2. 

5(  1.1) 

1.4(  0.7) 

0.2(  0.2) 

0 

0 

246. 9( 

3.2, 

241 

8( 

5.2) 

241. 2( 

3. 

1) 

262. 

6(12.7) 

240.4(14.1) 

239. 3(****) 

GRADUATED  H.S. 

1193 

136.783( 

3X) 

71. 9( 

1.7) 

15 

7( 

1.2) 

9.8( 

0. 

8) 

1. 

5(  0.3) 

1.0(  0.3) 

0.0(  0.0) 

0 

0 

260. 7( 

1.7) 

242 

4( 

3.0) 

244. 6( 

2. 

9) 

271. 

3(  7.9) 

251. 1(  9.6) 

SOME  EDUC  AFTER  H.S. 

781 

89,177( 

42) 

73. 9( 

1.9) 

14 

9( 

1.7) 

8.7( 

0. 

9) 

1. 

3(  0.3) 

1.2(  0.3) 

0.0(  0.0) 

0 

0 

272. 4( 

1.8) 

257 

5( 

2.7) 

251. 4( 

3. 

9) 

287 

0(  8.0) 

276.9(10.0) 

GRADUATED  COLLEGE 

1639 

189,833( 

4X) 

75. 8( 

1.4) 

13 

0( 

1  1) 

5.8( 

0 

6) 

4 

2(  0.8) 

l.K  0.3) 

0  ^{  0.0) 

0 

0 

278. 6( 

1.3) 

248 

5( 

2.7) 

248. 2( 

4 

5) 

282 

6(  3.3) 

251. 3(  7.2) 

UNKNOT 

403 

36,033( 

7X) 

41. 7( 

2,6) 

21 

6^ 

2  2) 

26. 8( 

1 

9) 

6 

8(  1.7) 

2.9(  1.0) 

0.3(  0.3) 

0 

0 

251. 3( 

4.0) 

234 

2( 

3.3) 

233. 5( 

3 

3) 

253 

2(  5.3) 

221.4(11.7) 

216. 1(****) 

*  CV  is  tho  coefficient  of  variation  for  the  sum  of  the  weishts. 
(****)  Standard  error  is  greater  than  99.9. 
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Tabl  16-76 


Weighted  Response  Percentages  and  Reading  Proficiency  Means  with  Standard  Errors 
Grade  8,  by  Parental  Ec!ucation,  for  MaJn  Focused-BIB  Reading  Samples 

PARENTS •  EDUCATION    (NAEP  ID:  PARED) 

N         WEIGHTED  N  (CV)*         NOT  HS  GRAD  HS  POST  HS  GRAD  COL  UNKNOWN  MISSING 

-    TOTAL  —  A391  A90,253(  XX)  7.A(  0.5)        27. 9(  0.9)        18. 2(  0.8) 


GENDER 
MALE 

FEMALE 


UNKNOWN 


4391 

490,253( 

IX) 

7 

.4( 

0 

.5) 

27 

.9( 

0.9) 

244 

.8( 

2 

.2) 

256 

.3( 

1,5) 

2234 

252,546( 

32) 

5 

.8( 

0 

.6) 

28 

.0( 

1,0) 

9157 

236 

.8( 

3 

.9) 

2^)9. 9  ( 

2.0) 

237,707( 

21) 

9 

.!( 

0 

.9) 

27 

.8( 

1.3) 

250 

.2( 

2 

.2; 

263 

.2( 

1.7) 

2691 

342,g65( 

22) 

5 

i>( 

0 

6) 

28 

7( 

1,3) 

766 

246 

9( 

3 

2) 

260 

7( 

1.7) 

73,869( 

5X) 

8 

3( 

1 

4) 

29 

K 

2.0) 

241 

8( 

5 

2) 

242 

4( 

3.0) 

709 

52,347( 

4Z) 

18 

9( 

2 

2) 

6( 

1.5) 

241 

2( 

3 

1) 

^44 

6( 

2.9) 

225 

21.073( 

8Z) 

7 

0( 

2 

2) 

16 

6( 

3.2) 

253 

9( 

9 

2) 

26o 

2( 

6.7) 

375 

36,42;( 

8Z) 

100 

0( 

0.0) 

0 

0( 

0.0) 

244 

8( 

2 

2) 

1193 

136.783( 

3Z) 

0 

0( 

0 

0) 

100. 

0( 

0.0) 

256. 

3( 

1.5) 

7»1 

89,177( 

4Z) 

0.0( 

0. 

0) 

0. 

0( 

0.0) 

1639 

189,833( 

4Z) 

0. 

0( 

0. 

0) 

0. 

0( 

0.0) 

403 

38,033( 

7Z) 

0. 

0( 

0 

0) 

0. 

0( 

0.0) 

38. 7(  1.3)         7.8(  0,5)  0  3 

268. 6(  1.4)      272. 7(  1.2)      242. 0(  2.0) 


16, 7(  0.8)  40, 5(  1.3)  9.0(  0.7)  0,3 

261. 2(  2.1)  266  1.8)  237. 6(  2.7) 

V.8(  1.1)  36  '  7)  6.5(  0.5)  0.2 

275. 3(  1.8)  280  0  248. 6(  2.3) 


RACE/ETHNICifY 

2691           342.965(  2Z)                   0.6)        28. 7(  1,3)        19. 2(  0.9)  *2.0(  1,6)  4  6(  0  4)              0  1 

,                      246. 9(  3.2)      260. 7(  1.7)      272. 4(  1.8)  278. 6(  1.3)  251. 3(  «.0) 

^^^^                                       766             73,869(  5X)           8.3(  1,A)        29. 1(  2.0)        18. 0(  2.1)  33. 5(  2.1)  11. 1(  1,6)               1  0 

aTCP*«Tr                                                     CO  o     .               ^^^-^^  ^-^^      2^2-*^  2           257. 5(  2.7)  248, 5(  2.7)  234, 2(  3.3) 

HISP/KIC                                  709             52.347(  4Z)          18. 9(  2.2)             6(  1.5)        U.8(  1.3)  21. 2(  1.9)  19. 5(  14)              0  1 

r.^„r^                                                                                241. 2(  3.1)      ^44. 6(  2.9)      251. 4(  3.9)  248. 2(  4,5)  233. 5(  3.3) 

^^^^                                      225             21,073(  8Z)           7.0(  2.2)        16. 6(  3.2)        10. 4(  2.1)  47. 9(  6.6)  18. 0(  3.1)  0.9 

282. 2(  6.6)  275. 9(  2.6)  243. 0(  6.7) 

PARENTAL  EDUCATION 

LESS  TtlAN  H.S.                        375             36,42;(  8Z)        100. 0(  0.0)         0.0(  0.0)         0.0(  0.0)  0.0(  0.0)  0,0(  0  0)              0  0 

244. 8(  2.2) 

GRADUAT.'D  H.S.                       1193           136,783(  3Z)           0.0(  0.0)      100. 0(  0.0)         o.O(  0.0)  0.0(  0.0)  0  0(  0,0)              C  0 

256. 3(  1.5) 

SOME  EDUC  AFTER  H.S.               7»1             89.177(  4Z)           0.0(  0.0)         0.0(  0.0)      100. 0(  0.0)  0.0(  0.0)  0.0(  0.0)              0  0 

268. 6(  1.4) 

GRADUATED  COLLEGE                  1639           189,833(4Z)           0.0(0.0)         0.0(0.0)         0.0(0.0)  100.0(0.0)  0.0(0.0)  0.0 

272. 7(  1.2) 

0(  0.0)  0.0(  0  0)  100. 0(  0  0)  0.0 
242. 0(  2  0) 


*  CV  is  the  coofficiont  of  variation  for  tho  sum  of  the  weights. 


435 


Table  16-77 


Weighted  Response  Percentages  and  Reading  Proficiency  Means  with  Standard  Errors 
Grade  12,  by  Gender,  for  Main  Focused-BIB  Reading  Samples 


GEKDER  OF  SUBJECT    (HAEP  ID:  SEX) 


WEIGHTED  H  (CV)*  MALfc 


FEMALE 


MISSING 


-    TOTAL  — 

4250 

521.501(  2X) 

47. 5( 

1.1) 

52. 5( 

1.1) 

0.0 

282. 9( 

1.1) 

291. 0( 

1.1) 

GEHDER 

MALE 

2002 

247,B66(  3Z) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0 

282. 9( 

1.1) 

FEMALE 

2246 

273.635(  3Z) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0 

291. 0( 

1.1) 

RACE/ETHNICITY 

WHITE 

2887 

385.848(  2Z) 

47. 1( 

1.4) 

52. 9( 

1.4) 

0.0 

288. 5( 

1.1) 

296. 2( 

1.2) 

BLACK 

748 

72,396(  6Z) 

45. 4( 

2.4) 

54. 6( 

2.4) 

0.0 

264. 4( 

2.5) 

275. 2( 

2.6) 

HISPANIC 

473 

41,8S3(  8Z) 

46. 7( 

2.3) 

53. 3( 

2.3) 

0.0 

257. 6( 

3.1) 

275. 3( 

3.2) 

OTHER 

142 

21.404(102) 

63. 5( 

4.9) 

36. 5( 

4.9) 

0.0 

287. 8( 

4.5) 

279. 9( 

5.4) 

PARENTAL  EDUCATION 

LESS  THAN  H.S. 

385 

39.295(  QX) 

40. 0( 

3.0) 

60. 0( 

3.0) 

0.0 

263. 9( 

4.0) 

276. 4( 

2.1) 

GRADUATED  H.S. 

1060 

128, 097 (  5Z) 

45. 7( 

2.0) 

54. 3( 

2.0) 

0.0 

273. 0( 

1.8) 

282. 5( 

1.7) 

SOME  EDUC  AFTER  H.S. 

1057 

127.961(  4Z) 

45. 0( 

1.8) 

55. 0( 

1.6) 

0.0 

285. 2( 

1.7) 

291. 3( 

1.5) 

GRADUATED  COLLEGE 

16A3 

21A,061(  3Z) 

51. 2( 

1.6) 

48. 8( 

1.6) 

0.0 

291. 0( 

1.5) 

301. 0( 

1.7) 

UNKNOWN 

97 

10,952(10Z) 

51. 3( 

6.8) 

48. 7( 

6.8) 

0.0 

253. 8( 

5.1) 

265. 4( 

9.0) 

*  CV  is  the  corffici«nt  of  variation  for  the  sum  of  the  woishts. 
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Table  16-78 


Weighted  Response  Percentages  and  Reading  Proficiency  Means  with  Standard  Er*.ors 
Grade  12,  by  Derived  Race/ethnicity,  for  Main  Focused-EIB  Reading  Samples 


DERIVED  RACE/ETHNICITY    (NAEP  ID:  DRACE) 


N 

WE.CGHTED  N  (CV)* 

WHITE 

BUCK 

—    TOTAL  — 

4250 

521,501(  2X) 

74 

.0( 

0 . 7  J 

293 

6( 

1.0) 

270. 3C 

1.6) 

GENDER 

MALE 

2002 

247.866(  3Z) 

73 

.4( 

1.2) 

13. 3( 

0.9) 

288 

5( 

1.1) 

264. 4( 

2.5) 

FEMALE 

2248 

273  fi^Sf  3t1 

74 

6( 

1.5) 

14. 4( 

1.0) 

2P6.2( 

1.2) 

275. 2( 

2.6; 

RACE/ETHNICITY 

WHITE 

2887 

385,848(  2X) 

100 

0( 

0.0) 

0.0( 

0.0) 

292 

6( 

1.0) 

hlACK 

748 

72,396(  6X) 

0 

0( 

0.0) 

100. 0( 

0.0) 

270. 3( 

2.6) 

HISPANIC 

473 

41,853(  8X) 

0 

0( 

0.0) 

0.0( 

0.0) 

uIHER 

142 

21.404(10X) 

0 

0( 

0.0) 

0.0( 

0.0) 

PARENTAL  EDUCATION 

LESS  THAN  H.S. 

385 

39.295(  8X) 

48 

6( 

4.2) 

13. 7( 

2  9) 

276 

8( 

3.1) 

265. 7( 

3.2) 

GRADUATED  H.S. 

1060 

128.097(  5X) 

73 

1( 

1.8) 

17. 0( 

1.6) 

283 

0( 

l.i) 

265. 3( 

?  9; 

SOME  EDUC  AFTER  H.S. 

1057 

127.961(  4X) 

74 

3( 

1.7) 

15. 9( 

1.3) 

293. 1( 

1.4) 

275. ^< 

2.8) 

GRADUATED  COLLEGE 

;^43 

214.061(  3X) 

81 

0( 

1.1) 

9 

0.7) 

299 

9( 

1.3) 

274. 1( 

2.8) 

UNKNOWN 

97 

10.952(10X} 

39. 

2( 

7.0) 

22. 6( 

4.0) 

262.3(12.7) 

255. 4( 

7.6) 

HISPANIC 

8.0(  0.5) 
267. 1(  2.4) 


7.9(  0.6) 

257. 6(  3.1) 

8.2(  0.8) 

275. 3(  3.2) 


0.0(  0.0) 

0.0(  0.0) 

100. 0(  0.0) 

267. 1(  2.4) 

0.0(  0.0) 


28. 8( 
266. 1( 

7.3( 
260. 7( 

6.3( 
273. 2( 

4.8{ 
271.il 
24. 0( 
254. 4( 


3.2) 
3.9) 
0.7) 
3.7) 
0.7) 
4.5) 
0.5) 
3.8) 
5.3) 
4.6) 


ASIAN 

2.2(  0.3) 
290. 3(  3.7) 


4.5(  0.5) 

292. 4(  5.3) 

2.0(  0.4) 

286. 2(  5.3) 


0.0(  0.0) 

0.0(  0.0) 

0.0(  0.0) 

77. 6(  5.8) 

290. 3(  3.7) 


2.0(  0.9) 

261. 1(  4.2) 

2.0(  0.5) 
275.3(14.0) 

2.7(  0.6) 

279. 2(  6.4) 

4.J(  0.6) 

307,. 9(  5.0) 

6.2(  3.2) 

282. 0(  7.9) 


AMER  IND  UNCLASSIFIED 


0.e(  0.3) 
267. 9(  7.4) 


0.9(  0.3) 
271.2(10.0) 

0.8(  0.4) 
264. 5(  8.9) 


0.0(  0.0) 

0.0(  0.0) 

0.0(  0.0) 

20. 6(  5.8) 

267. g(  7.4) 


1.9(  1.7) 
279.8(29.5) 

0.6(  0.3) 
270. 5( 

0-7( 
288. 2(  9.9) 

0.6(  0.3) 
257.0(12.5) 

5.9(  3.0) 
245.2(2C.0) 


5.6) 
0.3) 


O.K  0.1) 
246.0(28.5) 


0.](  0.1) 
235. o(****) 

0.0(  0.0.> 
268. 2(  6.8) 


0.0(  0.0) 

0.0(  0.0) 

0.0(  0.0) 

1.8(  1.3) 
246.0(28.5) 

0.0(  0.0) 
0.0(  0.0) 

O.K  0.1) 
268. 2(  6.8) 

O.K  0.1) 
235. 3(****) 

0.0(  0.0) 


MISSING 
0.0 

0.0 
0.0 

0.0 
0.0 
0.0 
0.0 

0.0 
0.0 
0.0 
0.0 
0.0 


*  CV  is  the  coeCCicient  of  variation  for  the  sum  of  the  weights. 
(****)  Standard  error  is  greater  than  99.9. 
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Table  16-79 


Weighted  Response  Percentages  and  Reading  Proficiency  Means  with  Standard  Errors 
Grade  12,  by  Parental  Education,  for  Main  Focused-BIB  Reading  Samples 


PAREHTS*  EDUCATION    (KAEP  ID:  PARED) 

N 

—    TOTAL  — 


GEKDER 
MALE 

FEMALE 

RACE/ETflNICITY 
WHI7E 

BUCK 

HISPANIC 

OTHER 

PAREKTAL  EDUCATION 
LESS  THAN  H.S. 

GRADUATED  H.S. 

SOME  EDUC  AFTER  H.S. 

GR  \DUATED  COLLEGE 

UNKNC.n; 


?fSIGHTED  N  (CV)* 


NOT  HS 


GRAD  HS 


POST  HS 


GRAD  COL 


424? 

520,364(  2Z} 

7.6( 

0.6) 

24. 6( 

1 

.0) 

24. 6( 

0.9) 

41 

.K 

1.5) 

271. 4( 

2.1) 

278. 2( 

1 

.1) 

288 

.6( 

1 

.2) 

295 

.9< 

1.2) 

1997 

247,126(  3X) 

6.4( 

0.7) 

23 

.7( 

1 

.3) 

7" 

3( 

1 

.1) 

44 

1.9) 

2245 

263. 9( 

4  0) 

273 

.0( 

1 

.8) 

1 

.  7  ) 

291 

.0( 

1.5) 

273,238(  3Z) 

8.6( 

0.3) 

25 

.4( 

1 

.2) 

25 

.o( 

1 

.1) 

38 

.2{ 

1.6) 

275. 4( 

2.1) 

282 

.5( 

1 

7) 

291 

.3( 

^ 

.5) 

301 

.0( 

1.7) 

2884 

385,571(  2Z) 

5.0( 

0.6) 

2- 

.3( 

1 

2) 

24 

.7( 

1 

0) 

45 

0( 

1.7) 

746 

276. 8( 

3.1) 

283 

.0( 

1 

4) 

293 

.1( 

1 

4) 

299 

9( 

1.3) 

72,150(  6X) 

10. 2( 

1.5) 

30 

K 

2 

0) 

28 

.2( 

1 

7) 

28 

K 

2.6) 

471 

265. 7( 

3.2) 

265 

.3( 

2 

9) 

275 

.3( 

2 

8) 

274 

K 

2.8) 

41,512(  8Z) 

27. 3( 

2.2) 

22 

4( 

1 

8) 

19 

3( 

1 

5) 

24 

7( 

2.3) 

141 

266. 1( 

3.9) 

260 

7( 

3 

7) 

273 

2( 

4 

5) 

271 

K 

3.8) 

21.131(10Z) 

7.2( 

3.3) 

16. 1( 

2 

9) 

21 

4( 

3 

9) 

47 

9( 

6.4) 

270. 2( 

5.7) 

274 

1(10 

9) 

280 

7( 

5 

1) 

295 

2( 

5.5) 

385 

39.295(  8Z) 

100. 0( 

0.0) 

0 

\ 

0. 

0) 

0 

0( 

0 

0) 

0. 

0( 

0  0) 

271. 4( 

2.1) 

1069 

128,097(  5Z) 

0.0( 

0.0) 

100 

0( 

0. 

0) 

0 

0< 

0. 

0) 

0. 

0( 

0.0) 

278 

2( 

1. 

1 . 

1057 

127,961'  4Z) 

0.0( 

0.0) 

0 

0( 

0. 

0) 

100 

0( 

0. 

0) 

0. 

0( 

0.0) 

1643 

288 

6( 

1. 

2) 

214.061(  3Z) 

0.0( 

0.0) 

0 

0( 

0. 

0) 

0 

0( 

0. 

0) 

100. 

0( 

0.0) 

295. 9( 

1.2) 

97 

10,952(10Z) 

Q.0( 

0.0) 

0. 

0( 

0. 

0) 

0. 

0( 

0. 

0) 

0.0( 

0.0) 

UNKNOWN 

2.1(  0.2) 
259. 5(  6.0) 


2.3(  0.4) 

253. 8(  5.1) 

2.0(  0.3) 

265. 4(  9.0) 


l.K  0.2) 
262.3(12.7) 

3.4(  0.7) 
255. 4(  7.6) 

6.3(  1.3) 
254. 4(  4.6) 

7.3(  2.3) 
266.6(11.8) 


0.0(  0.0) 

0.0(  0.0) 

0.^  0.0) 

0.0(  0.0) 

100. 0(  0.0) 

259. 5(  6.0) 


MISSING 
0.2 


0.3 
0.1 


0.1 
0.3 
0.8 
1.3 

0.0 
0.0 
0.0 
0.0 
0.0 


*  CV  is  th«  coofficient  oC  variation  fo-  th«  sum  of  the  weights. 
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Table  16-80 


Weighted  Response  Percentages  and  Writing  Prof iciencjr*  Means  with  Standard  Errors 
Grade  4,  by  Gender,  for  Main  Writing  Samples 


GENDER  OF  SUBJECT    (NAEP  ID;  SEX) 

N     WEIGHTED  M  (CV)**     MALE  FEMALE  MISSTWG 


—TOTAL- 

6677 

i ,270,697( 

IZ) 

SO.^C 

0 

9) 

49 

.6( 

0 

9) 

0 

.0 

1 

3) 

197 

.5( 

1 

3) 

GENDER 

MALE 

3356 

640.X62( 

2X) 

100. 0( 

f/ 

0) 

0 

0( 

0 

0) 

0 

.0 

184. 4r 

3) 

FEMALE 

3321 

630,535( 

2X) 

o.oc 

0 

0) 

100 

0( 

0 

0) 

0 

0 

197 

5( 

1 

3) 

RACE/ETHNICITY 

WHITE 

3707 

881,758( 

IX) 

SO./i( 

1 

1) 

49 

6( 

1 

1) 

0 

0 

191. 1( 

1 

6) 

204 

3( 

1 

7) 

BLACK 

1206 

ig2,142( 

2X) 

A5.3( 

1. 

7) 

54 

7( 

1 

7) 

0 

0 

161. 9( 

3 

3) 

174 

5( 

1 

8) 

HISPANIC 

UOS 

1AA.578< 

2X) 

56. 9( 

1. 

3) 

43 

K 

1 

3) 

0 

0 

172. 1( 

2. 

0) 

186 

2' 

2 

8) 

OTHER 

3S9 

S2.218( 

5X) 

50. 2( 

2. 

6) 

49 

8( 

2 

6) 

0 

0 

183. 9( 

4. 

9) 

204 

2( 

4 

4) 

PARENTAL  EDUCATION 


LESS  THAN  HS 

358 

62,539( 

7X) 

46 

3( 

2 

9) 

53. 7( 

2 

9) 

0 

0 

172 

9( 

4 

4) 

181. 9( 

3 

7) 

GRADUATED  HS 

978 

194,058( 

4X) 

49 

5( 

1 

8) 

5C.5( 

1 

8) 

0 

0 

178 

K 

2 

Z) 

194.3^ 

2 

5) 

SOME  EDUo  AFTER  HS 

508 

99,842( 

5X) 

49 

K 

2 

1) 

50. 9( 

2 

.) 

0 

0 

i35 

7( 

3 

3) 

208. 2( 

3 

3) 

GRADUATED  COLLEGE 

2403 

h79.780( 

*X) 

52 

2( 

1 

2) 

47. 8( 

1 

2) 

0 

0 

194 

0( 

1 

5) 

208. 0( 

1 

8) 

UNKNOWN 

2385 

428,174( 

«) 

49 

5( 

1 

2) 

50. 5( 

1 

2) 

0 

c 

±75 

4( 

1 

9) 

188. 0( 

2 

0) 

*  Average  response  method  proficiency  scor- 

**  CV  is  the  coefficient  of  variation  for  the  sun  of  the  weights. 
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Table  16-81 


Weighted  Response  Percentages  and  Writing  Proliciency*  Means  with  Standard  Errors 
Grade  4,  by  Derived  Race/ethnicity,  for  Main  Writing  Samples 


DERIVED  RACE/ETHNICITY    (NAEP  ID:  DRACE) 

H     WEIGHTED  N  (CV)**     WHITE  BLACK  HISPANIC  ASIAN  AMER  IND     UNCLASSIFIED  MISSING 


—TOTAL- 

6679 

1, 270. 904 ( 

IX) 

69 

4( 

0. 

4) 

15. 

i( 

0 

3) 

11 

4( 

0 

3) 

1 

9( 

0 

2) 

2. 

1( 

0 

2) 

0 

K  0 

0) 

0.0 

197 

6( 

1. 

3) 

168. 

8( 

1 

9) 

178 

2( 

2 

0) 

20£* 

0( 

6. 

9) 

181 

0( 

4 

5) 

188 

3(13 

3) 

GENDER 

MALE 

3357 

640,295( 

2:) 

6S 

4( 

0. 

7) 

13 

6( 

0 

5) 

12 

0 

4) 

1 

7( 

0. 

2) 

2 

3( 

0 

2) 

0 

0(  0 

0) 

0.0 

191 

1( 

1. 

6) 

161 

9( 

3 

2) 

172 

1( 

2 

0) 

197 

7( 

6 

7) 

173 

4( 

7 

2) 

187 

2(34 

3) 

FEMALE 

3322 

630, 610 ( 

2X) 

69 

3( 

0. 

7) 

16 

7( 

0 

6) 

9 

9( 

0 

4) 

2 

0( 

0 

3) 

2 

0( 

0 

2> 

0 

K  0 

1) 

0.0 

204 

3( 

X  • 

7> 

■  ^  • 

5( 

1 

8) 

186 

2( 

2 

8) 

218 

7( 

9 

2) 

190 

1( 

5 

0) 

183 

6(17 

2) 

RKr^^/ETHNICITY 

WHITE 

3707 

881,758( 

IX) 

100 

0( 

0 

0) 

0 

or 

0 

0) 

0 

0( 

0 

0; 

0 

0^ 

0 

0) 

0 

0( 

0 

0) 

0 

0(  0 

0) 

0.0 

197 

.6( 

1 

3) 

BLACK 

1207 

192,275( 

2X) 

0 

.0( 

0 

0) 

100 

0( 

0 

0, 

0 

.or 

0 

■  v) 

0 

0( 

0 

0) 

0 

0( 

0 

.0) 

0 

0(  0 

0) 

0.0 

i6e 

8( 

1 

9) 

HISPANIC 

1406 

144,65j( 

2X) 

0 

0( 

0 

0) 

0 

.0( 

0 

.0) 

100 

.0( 

0 

.C) 

0 

.0( 

0 

0) 

0 

G( 

0 

0) 

0.0(  0 

0) 

0.0 

178 

.?( 

2 

.0) 

OTHER 

359 

52,218( 

5X) 

0 

.0( 

0 

0) 

0 

.0< 

0 

.0) 

0 

.;( 

0 

.0) 

46 

1( 

\ 

1) 

^2 

2( 

4 

0) 

1 

8(  1 

0) 

0.0 

209 

.0( 

6 

S) 

Idl 

0( 

4 

.5) 

188 

.3(13 

.3) 

PARENTAL  EDUCATION 


LESS  THAN  HS 

358 

62. 539 ( 

7X) 

60. 3( 

2. 

5) 

19. 6( 

2 

5) 

16 

9( 

2 

1) 

1 

6(  1 

1) 

1 

6(  0 

6) 

0.0(  0 

0) 

0.0 

183. 3( 

4. 

2) 

165. 5( 

6 

5) 

174 

6( 

6 

9) 

173 

8(61 

6) 

132 

4(12 

I) 

GRADUATED  HS 

978 

194,058( 

4X) 

71. 1( 

1. 

3) 

17. 8( 

1 

3) 

8 

1( 

0 

8) 

0 

8(  0 

3) 

2 

2(  0 

3) 

0.0(  0 

0.0 

191. 8( 

2. 

1) 

166. 9( 

T) 

17V 

9( 

4 

1) 

229 

0(24 

8) 

183 

6(10 

0) 

SOME  EDUC  AFTER  HS 

509 

99,917( 

5X) 

75. 1( 

2 

0) 

10. 8( 

1 

6) 

10 

3( 

1 

1) 

1 

4(  0 

5) 

2 

2(  0 

6) 

0.3(  0 

3) 

0.0 

206. 6( 

3 

1) 

176. 2( 

7 

8) 

192 

4( 

5 

2) 

231 

5(ia 

^) 

205 

2(13 

6) 

178. 8<  5 

7) 

GRADUATED  COLLEGE 

2404 

479,913( 

4X) 

73. 3( 

1 

2) 

14. 0( 

0 

^) 

9 

0( 

0 

6) 

1 

8(  G 

3) 

1 

8(  0 

3) 

O.K  0 

o> 

0.0 

207. 2( 

1 

5) 

174. 5( 

1. 

2) 

186 

9( 

3 

2) 

219 

1(  6 

2) 

187 

6(  8 

2) 

189.2(18 

4) 

UNKNOWN 

2335 

428,174( 

4X) 

64. 5( 

1 

0) 

15. 2( 

0 

9) 

15 

0( 

0 

7) 

2 

7C  0 

4) 

2 

6(  0 

3) 

O.K  0 

0) 

0  0 

188. 2( 

1 

8) 

163. 4( 

2 

.3) 

171 

C( 

2 

6) 

199 

0(  7 

C) 

.172 

9(  6 

0) 

195.9(37 

1) 

*  Avorage  response  mothod  proficiency  score 

CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 
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Table  16-82 

Weighted  Response  Percentages  and  Writing  Proficiency*  Means  with  Standard  Errors 
Grade  4,  by  Parental  Education,  for  Main  Writing  Sarcples 


PARENTS'  EDUCATION    (NAEP  ID:  PARED) 

N     WEIGHTED  N  (CV)**     NOT  HS 

—TOTAL- 


GENDER 
MALE 

FEMALE 

RACE/ETHNICITY 
WHITE 

BLACK 

HISPANIC 

OTHER 

PARENTAL  EDUCATION 
LESS  THAN  HS 

GRADUATED  HS 

SOME  EDUC  AFTER  HS 

GRADUATED  COLLEGE 

UNKNOWN 


\D  HS 


POST  HS 


GRAD  COL 


UNKNOWN 


6679 

1,264,600( 

IZ) 

4 

.9( 

0 

4) 

15 

.3( 

0 

.6) 

7 

.9( 

0 

4) 

37 

.9( 

1 

4) 

33 

.9( 

1 

1) 

177 

.7( 

2 

.9) 

186 

.3( 

1 

.9) 

202 

1( 

2 

5) 

200 

.7( 

1 

.3) 

181 

.7( 

1 

.1) 

3357 

636,711( 

2Z) 

4 

.5( 

0 

5) 

15 

1( 

0 

7) 

7 

7( 

0 

6) 

39 

4( 

1 

5) 

33 

.3( 

1 

1) 

172 

9( 

4 

4) 

178 

1( 

2 

2) 

195 

7( 

3 

3) 

194 

0( 

1 

4) 

175 

•  4( 

\ 

9) 

3322 

627,890( 

25) 

5 

3( 

G 

4) 

15 

6( 

0 

7) 

8 

K 

0 

5) 

36 

5( 

1 

6) 

34 

5( 

1 

4) 

181 

9( 

3 

7) 

194 

3( 

I 

5) 

208 

2( 

3 

3} 

208 

0( 

1 

8) 

188 

0( 

2 

0 ) 

3707 

878,914( 

IZ) 

4 

3( 

0 

4) 

15 

7( 

0 

7) 

8 

5( 

0 

5) 

40 

0( 

1 

7)  ' 

31 

4X 

1 

3) 

183 

3( 

4 

2) 

191 

8( 

2 

1) 

206 

6( 

3 

1) 

207 

2( 

1 

5) 

188 

2( 

1 

8) 

1207 

189.810( 

2Z) 

6 

5( 

1 

0) 

18 

2( 

1 

2) 

5 

7( 

0 

8) 

35 

4( 

1 

9) 

34 

2( 

2 

2) 

165. 5( 

6 

5) 

166 

9( 

3 

7) 

176 

2( 

7 

8) 

174 

5( 

4 

7.\ 

163 

4( 

2 

3) 

1406 

143,751( 

3Z) 

7 

3( 

0 

8) 

11 

0( 

1 

2) 

7 

2( 

0 

7) 

29 

9( 

1 

1% 

44 

6( 

1 

4) 

174 

6( 

6 

9) 

177 

9( 

4 

1) 

192 

4( 

5 

2) 

186 

9( 

3 

2) 

171 

0( 

2 

6) 

359 

5?.,126( 

5Z) 

3 

8( 

1 

3) 

11 

1( 

2 

2) 

7 

3( 

1 

6) 

33 

9( 

2 

8) 

43 

9( 

3 

1) 

163 

0(16 

6) 

195 

5(11 

2) 

212 

9( 

9 

3) 

203 

2( 

4 

9) 

186 

K 

4 

5) 

358 

62,539( 

7Z) 

100 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

or 

0 

0) 

0 

0( 

0. 

0) 

177 

7( 

2 

9) 

978 

194,058( 

4Z) 

0 

0( 

0 

0) 

100 

0( 

0 

0) 

0 

0( 

0. 

0) 

0 

0( 

0 

0) 

0 

0( 

0. 

0) 

186 

3( 

1 

9) 

509 

99,917( 

5Z) 

0 

0( 

0. 

0) 

0 

0( 

0 

0) 

100. 

0( 

0. 

0) 

0. 

0( 

0 

0) 

0 

0( 

0. 

0) 

202. 1( 

2. 

5) 

2404 

479.913( 

4Z) 

0 

0( 

0. 

0) 

0 

0( 

0 

0) 

0. 

0( 

0. 

0) 

100 

0( 

0. 

0) 

0 

0( 

0. 

0) 

200. 

7( 

1. 

3) 

2385 

428,  W4( 

4t) 

0 

0( 

0 

0) 

0 

0( 

0. 

0) 

0. 

0( 

0. 

0) 

0. 

0( 

0. 

0) 

100. 

0( 

0. 

0) 

MISSING 
0.5 

0.6 

\4 

0.3 
1.3 
0.6 
0.2 

0.0 
0.0 
0.0 
0.0 

0.0 


181. 7(  1.1) 


*  Average  response  method  proficiency  score 

**  (TV  is  the  coefficient  of  variation  for  the  sum  of  the  weights, 
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Table  16-83 


Weighted  Response  Percentages  and  Writing  Proficiency*  Means  with  Stand&rd  Errors 

Grade  8,  by  Gender,  for  Main  Writing  Samples 

GENDER  OF  SUBJZCT    (NAEP  ID:  SEX) 

N     WEIGHTED  N  (CV)**     MALE  FEMALE  MISSING 


—TOTAL- 

6525 

743, 791 ( 

IZ) 

51. 1( 

0.8) 

48. 9( 

0.8) 

0. 

0 

200. 9( 

1.4) 

218. 5( 

1.2) 

GENDER 

MALE 

329A 

379,715( 

2Z) 

100. 0( 

0.0) 

0.0( 

0.0) 

0. 

0 

200. 9( 

1.4) 

FEMALE 

3231 

364,076( 

2Z) 

0.0( 

0.0) 

100. 0( 

0.0) 

0. 

0 

218. 5( 

1.2) 

RACE/ETHNICITY 

WHITE 

4143 

537,337( 

2Z) 

50. 3( 

0.9) 

49. 2( 

0.9) 

0. 

0 

207. 2( 

1.6) 

225. 0( 

1.1) 

BLACK 

1119 

108,142( 

3Z) 

49. 9( 

2.0) 

50. 1( 

2.0) 

0. 

0 

179. 4( 

2.5) 

195. 7( 

2.6) 

HISPANIC 

966 

69, 963 ( 

3Z) 

53. 6( 

2.1) 

46. 4( 

2  1) 

0 

0 

184. 5( 

2.6) 

201. 5( 

3.4) 

OTHER 

297 

28,349( 

4Z) 

53. 4( 

2.3) 

46. 6( 

2.3) 

0 

0 

205. 0( 

4.2) 

221. 7( 

5.7) 

PARENTAL  EDUCATION 


LESS  THAN  HS 

603 

59,584( 

6Z) 

39. 8( 

2.3) 

60. 2( 

2 

3) 

0. 

0 

185. 2( 

4.1) 

204. 1( 

2 

4) 

GRADUATED  HS 

1706 

198,188( 

3Z) 

52. 0( 

1.4) 

48. 0( 

1 

4) 

0. 

0 

196. 3( 

1.9) 

214. 6( 

2 

4) 

SOME  EDUC  AFTER  i.S 

1178 

135,639( 

^Z) 

47. 9( 

1.8) 

52. 1( 

1 

8) 

0. 

0 

207. 3( 

2.6) 

224. 2( 

2 

0) 

GRADUATED  COLLEGE 

2444 

292,035( 

3Z) 

52. 9( 

1.1) 

47. 1( 

1 

1) 

0. 

0 

208. 3( 

2.1) 

225. 8( 

1 

9) 

UNKNOWN 

573 

56,300( 

6X) 

57. 8( 

2.3) 

42. 2( 

2 

3) 

0. 

0 

179. 3( 

3.0) 

196. 7( 

3 

6) 

*  Averase  response  method  proficiency  score 

**  CV  is  the  coefficient  of  variation  i.or  the  »um  of  the  weights. 
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Table  16-84 


Weighted  Response  Percentages  and  Writing  Proficiency*  Means  with  Standard  Errors 
Grade  8,  by  Derived  Race/ethnicity,  for  Main  Writing  Samples 


DERIVED  RACE/ETHHICITY    (NAEP  ID:  DRACE) 

N     WEIGHTED  N  (CV)**     WHITE  BLACK  HISPANIC  ASIAN  AMER  IND     UNCLASSIFIED  MISSING 


—TOTAL- 

6525 

743,791( 

IZ) 

72 

2( 

0.5) 

14 

5( 

0.4) 

9.4( 

0.2) 

2.5( 

0.2) 

1.2( 

0. 

2) 

0. 

K 

0. 

0) 

0 

.0 

71S 

0( 

1.0) 

X87 

5( 

2.2) 

192. 4( 

2.1) 

222. 5( 

5.0) 

194. 1( 

5. 

1) 

193. 

8(17. 

5) 

GENDER 

MALE 

329A 

379, 715^ 

21) 

71 

9( 

0.9) 

14 

2( 

0.7) 

9.9( 

0.5) 

2.5( 

0.2) 

1.4( 

0. 

2) 

0. 

K 

3. 

0) 

0 

.0 

207 

2( 

1.6) 

X79 

4( 

2.5) 

184. 5( 

2.6) 

216. 3( 

5.7) 

186. 1( 

o . 

6) 

191. 

2(24. 

7) 

FEMALE 

3231 

364,076( 

21) 

72 

6( 

0.8) 

14 

9( 

0.6) 

8.9( 

0.4) 

2.5( 

0.2) 

1.0( 

0. 

2) 

0. 

K 

0. 

1) 

0 

.0 

225 

0( 

1.1) 

195 

7( 

2.6) 

201. 5( 

3.4) 

228. 9( 

7.4) 

206. 3( 

8 

5) 

195. 

6(19. 

3) 

RACE/ETHNICITY 

WHITE 

4143 

537,337( 

21) 

100 

0( 

0.0) 

0 

0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0/ 

0.0( 

0. 

0) 

0. 

0( 

0. 

0) 

0 

.0 

216 

0( 

1.0) 

BLACK 

1119 

108,142( 

3.) 

0 

0( 

0.0) 

100 

0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0( 

0. 

0) 

0. 

0( 

0. 

0) 

0 

0 

187 

5( 

2.2) 

HISPANIC 

966 

69, 963 ( 

3X) 

0 

0( 

0.0) 

0 

0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0. 

0) 

0. 

0( 

0. 

0) 

0 

.0 

192. 4( 

2.1) 

OTHER 

297 

28,349( 

4X) 

0 

0( 

0.0) 

0 

0( 

0.0) 

0.0( 

0.0) 

65. 8( 

4.4) 

31. 9( 

4. 

2) 

2. 

3( 

1. 

2) 

0 

0 

222. 5( 

5.0) 

194. 1( 

5. 

1) 

193. 

8(17. 

5) 

PARENTAL  EDUCATION 


LESS  THAN  HS 

603 

59, 584 ( 

6Z) 

59. 6( 

2.5) 

15. 2( 

1.7) 

23. Z( 

1. 

8) 

1.2(  0.5) 

0.8(  0.5) 

0 

0(  0.0) 

0. 

0 

203. 3( 

2.8) 

183. 5( 

6.1) 

187. 8( 

4. 

8) 

204.5(13.2) 

182.0(16.4) 

GRADUATED  HS 

1706 

198, 188 ( 

3X) 

72. 1( 

1.3) 

16. 7( 

1.1) 

8.6( 

0. 

6) 

1.2(  0.3) 

1.4(  0.3) 

0 

K  0.1) 

0. 

0 

211. g( 

2.2) 

182. 7( 

4.9) 

192. 2( 

3. 

7) 

214.9(10.9) 

194.7(11.5) 

209 

8(22.9) 

SOME  EDUC  AFTER  HS 

1178 

135,639( 

4Z) 

75. 7( 

1.0) 

14. 0( 

0.8) 

6.9( 

0. 

6) 

1.6(  0.4) 

1.8(  0.4) 

0 

0(  0.0) 

0. 

0 

220. 7( 

1.8) 

200. 4( 

4.0) 

199. 5( 

6. 

2) 

222.6(10.6) 

2)7. 1(  7.3) 

183 

5(22.6) 

GRADUATED  COLLEGE 

2444 

292,035( 

3Z) 

78. 5( 

1.0) 

11. 7( 

0.8) 

5.4( 

0. 

4) 

3.6(  0.4) 

0.6(  0.2) 

0 

0(  0.0) 

0. 

0 

220. 7( 

1.5) 

)90.7( 

2.6) 

204. 0( 

3. 

9) 

230. 8(  7.5) 

203.5(14.3) 

240 

8(11.0) 

UNKNOWN 

573 

56,300( 

6Z) 

46. 1( 

3.2) 

22. 1( 

1.8) 

24. 0( 

2. 

1) 

5.1(  0.8) 

2.6(  0.8) 

0 

X(  0.1) 

0. 

0 

195. 5( 

3.*.; 

i?5.5( 

t.7) 

178. 8( 

3, 

9) 

202. 7(  7.1) 

164.9(13.0) 

164 

4(16.5) 

*  Average  response  method  proficiency  score 

CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 
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Table  16-85 

Weighted  Response  Percentages  and  Writing  Proficiency*  Mbans  with  Standard  Errors 
Grade  8,  by  Parental  Education  for  Main  Writing  Samples 


PARENTS'  EDUCATION    (HAEP  ID;  PARED) 

N     WEIGHTED  N  (CV)**      NOT  HS 


"TOTAL- 


6525       741.7<.6(  IX) 


6.0(  0.4) 
1S6.6(  2.1) 


GRAO  HS 

26. 7(  0.8) 
205. 1(  1.8) 


POST  HS 

18. 3(  0.6) 
216. 1(  1.6) 


GRAO  COL 


39. AC  1. 
216. 5(  1. 


UNKNOWK 

7.6(  0.5) 
166. 6(  2.2) 


MISSING 
0.3 


GENDER 
MALE 

FEMALE 


3294 


3231 


378,715(  21) 
363,032(  2Z) 


6.3(  0.5) 

185. 2(  A.l) 

9.9(  0.6) 

204. 1(  2.4) 


27. 2(  1.1) 
196. 3(  1.9) 

26. 2(  1.0) 
214. 6<  2.4) 


17. 2(  0.9) 

207. 3(  2.6) 

19. 5(  0.7) 

224. 2(  2.0) 


40. 8(  1.1) 

208. 3<  2.1) 

37. 9(  1.3) 

225. 6(  1.9) 


8.C(  0.5) 

179. 3(  3.0) 

6.5(  0.6) 

196. 7(  3.6) 


0.3 


0.3 


RACE/ETHNICITY 


WHITE 

4143 

536,319( 

2X) 

6 

6( 

0.5) 

26. 

6( 

1.0) 

'.9 

K 

0. 

7) 

42. 

8( 

1. 

2) 

\ 

8( 

0. 

5) 

0 

.2 

203 

3( 

2.8) 

211 

9( 

2.2) 

220 

7( 

1. 

8) 

220. 

7( 

1. 

5) 

195 

5( 

3. 

4) 

BLACK 

1119 

107,806( 

31) 

8 

4( 

0.8) 

30. 

6( 

1.7) 

17 

6( 

1. 

1) 

31. 

8( 

2. 

0) 

11 

5( 

0. 

9) 

0 

.3 

183 

5( 

6.1) 

182. 

7< 

4.9) 

200 

4( 

4. 

0) 

190. 

7( 

2. 

6) 

175 

5( 

4. 

7) 

HISPANIC 

966 

69,604( 

3X) 

19 

ec 

1.6) 

24. 

6( 

1.7) 

13 

4( 

1. 

2) 

22. 

8( 

1. 

8) 

19 

4( 

1. 

9> 

0 

.5 

187 

ec 

4.8) 

)32. 

2( 

3.7) 

199 

5( 

6 

2) 

204. 

0( 

3. 

9) 

178 

li( 

3. 

OTHER 

297 

28,018( 

4X) 

4 

2( 

1.4) 

H 

7( 

3.0) 

16 

7( 

2 

4) 

44. 

6( 

3. 

7) 

15 

7( 

2. 

0) 

1 

.2 

195 

6(11.5) 

204 

1( 

8.8) 

.:i3 

9( 

5 

7) 

226. 

9( 

7. 

2) 

189 

4( 

6. 

2) 

PARENTAL  EDUCATION 

LESS  THAN  HS 

603 

59.5P4( 

6X) 

100 

0( 

0.0) 

0 

0( 

0.0) 

0 

0( 

0 

0) 

0. 

0( 

0. 

0) 

0 

0( 

0. 

0) 

0 

.0 

196 

6( 

2.1) 

GRADUATED  HS 

1706 

198, 188 ( 

3X) 

0 

0( 

0.0) 

100 

0( 

0.0) 

0 

0( 

0 

0) 

0. 

0( 

0 

0) 

0 

0( 

0 

0) 

0 

.0 

205 

K 

1.8) 

SOME  EDUC  AFTER  HS 

1178 

135,639( 

4X) 

0 

0( 

0.0) 

0 

0( 

0.0) 

100 

0( 

0 

0) 

0. 

0( 

0 

0> 

0 

0( 

0. 

0) 

0 

.0 

216 

1( 

1 

6) 

GRADUATED  COLLEGE 

2^44 

292,035( 

3X) 

0 

0( 

0.0) 

U 

0( 

0.0) 

0 

0( 

0 

0) 

100. 

0( 

0 

0) 

0 

0( 

0. 

0) 

0 

.0 

216. 

5( 

1 

5) 

UNKNOWN 

5/3 

56,300( 

6X) 

0 

0( 

0.0) 

0 

0( 

0.0) 

0 

0( 

0 

0) 

0. 

0( 

0 

0) 

100 

0( 

0 

0) 

0 

.0 

166 

.6( 

2. 

2) 

*  Aversse  responsa  method  proficiency  score 

**  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weishts. 
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Table  16-86 


Weighted  Response  Percentages  and  Writing  Proficiency*  Means  with  Standard  Errors 
Grade  12,  by  Gender,  for  Main  Writing  Samples 


GANDER  OF  SUBJECT    (KAEP  ID: 


N     WEIGETED  N  (CV)**  MALE 


F£^iAL£ 


MISSIKG 


""TOTAL— 

729,B19( 

2Z) 

47   7  f 

1    9  1 

n  n 

212. 5( 

1.4) 

234. 6( 

1.6) 

HALE 

347,787( 

3Z) 

1  nn  n  / 

n  n  1 

n  n  / 

n  n  1 

n  n 

382,032(  3Z/ 

0 . 0( 

0.0) 

1  nn  n( 

0.0) 

CO 

9^  A    fi  / 

1    0 1 

D  APP  /FT  H  WTP  TTV 

WHITE 

546,476( 

2Z) 

•1/  • 

1  L\ 

<(9  / 

1    A 1 

n  n 

91  o  n  f 
^lo . U  V 

1  71 

9^1  A/ 

1 

1 .  o  / 

1095 

103. 670 ( 

4Z) 

46. 6  ( 

2  1) 

53. 2( 

2  1) 

0.0 

191. 9( 

2.6) 

206! 5( 

3.4) 

HISPANIC 

612 

52,B02( 

5Z) 

46. 5( 

3.0) 

51. 5( 

3.0) 

0.0 

193. 2( 

4.3) 

216. 0( 

4.5) 

OTHER 

IQk 

24,671( 

8Z) 

:>3.3C 

4.6) 

46. 7( 

4.6) 

0.0 

216. 1( 

6.6) 

226. 1( 

6.9) 

PARENTAL  EDUCATIC5 

LESS  THAN 

541 

54, 507 ( 

7Z) 

41. 9( 

2.2) 

56. 1( 

2.2) 

O.C 

198. 1( 

3.6) 

216. 6( 

4.5) 

GRADUATED  HS 

152A 

179,919( 

4Z) 

49. 3( 

1.6) 

50. 7( 

1.6) 

0.0 

208. 5( 

2.5) 

229. 0( 

2.7) 

SOME  EDUC  AFTER  HS 

1475 

175,858C 

3X) 

44. 1( 

1.6) 

55. 9( 

1.6) 

0.0 

215. 6( 

2.6) 

235. 7( 

2.3) 

GRADUATED  COLLEGE 

2356 

301,753( 

3Z) 

49. 1( 

1.6) 

50. 9( 

1.8) 

0.0 

218. 1( 

1.6) 

243. 4( 

2.6) 

UNKNOWN 

U4 

14,413(12Z) 

62. 5( 

4.5) 

37. 5( 

4.5) 

0.0 

178. 6( 

7.7) 

186. 5( 

9.3) 

*  Average  response  method  proficiency  score 

**  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 
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Table  16-87 


Weighted  Response  Percentages  and  Writing  Proficiency*  Means  with  Standard  Errors 
Grade  12,  by  Derived  Rsce/ethnicity ,  for  Main  Writing  Samples 


DERIVED  RACE/ETHNICITY    (HAE?  ID:  DRACE) 

H     WEIGHTED  N  (CV>**  WHITE 


BLACK 


BISFAHIC 


ASIAN 


AMER  LMD  UlfCLASSIFIED 


HISSIHG 


—TOTAL- 

6069 

729,819( 

2X) 

75. 

2( 

0.7) 

1A.2( 

0.^^ 

7.2( 

0. 

3) 

2. 

8( 

0.3) 

0.5(  0. 

.1) 

C. 

,1(  0.0) 

0 

.0 

230. 

5( 

1.5) 

200. 7( 

2../ 

204. 9( 

3. 

,5) 

225. 

K 

5.4) 

212.1(17, 

.0) 

214. 

,8(34.1) 

GENDER 

K  0.:) 

KALE 

2934 

3A7,787( 

3X) 

7A, 

.9( 

1.0) 

1A.0( 

1.0) 

7.4( 

0, 

.5) 

3. 

0( 

0.4) 

0.7(  0. 

.2) 

0, 

0 

.0 

218. 

.0( 

T.7) 

191. 9( 

2.6) 

193. 2( 

4, 

.3) 

220. 

7( 

5.8) 

209.2(18, 

.5) 

203, 

.6(50.3) 

FEMALE 

3135 

382,032( 

3X) 

75, 

.A( 

0.9) 

14. 5( 

0.6) 

7.1( 

0. 

.6) 

2. 

6( 

0.4) 

0.4(  0. 

.2) 

0, 

.0(  0.0) 

0 

.0 

2A1. 

.8( 

1.8) 

208. 5( 

3.4) 

216. 0( 

4. 

.5) 

229. 

8( 

8.2) 

216.7(23. 

.6) 

RACE/ETHNICm 

.0) 

.0(  0.0) 

WHITE 

A178 

J48,A76( 

2X) 

100  < 

.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0 

.0) 

0. 

.0( 

0.0) 

0.0(  0 

0. 

0 

.0 

230 

.5( 

1.5) 

BLACK 

1095 

103,870( 

AX) 

0. 

.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0 

.0) 

0. 

.0( 

0.0) 

0.0(  0 

.0) 

0 

.0(  0.0) 

Q 

.  0 

200. 7( 

2.3) 

HISPANIC 

612 

S2,802( 

5X) 

0 

.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0 

.0) 

0. 

.0( 

0.0) 

0.0(  0 

.0) 

0, 

.0(  0.0) 

0 

.0 

204. 9( 

3 

.5) 

OTHER 

18A 

24,671( 

BX) 

0 

.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0 

.0) 

81. 

.8( 

4.2) 

16. 2(  3 

.9) 

2 

.0(  1.1) 

0 

.0 

225. 

.1( 

5.4) 

212.1(17 

.0) 

214 

.8(34.1) 

PARENTAL  EDUCATION 


LESS  THAN  HS 

541 

54,507( 

7X) 

55, 

K 

3. 

1) 

18. 

7( 

220. 

2( 

4. 

3) 

189. 

4( 

GRADUATED  HS 

1524 

179,919( 

4X) 

74, 

K 

1. 

6) 

16. 

7( 

225. 

.2( 

2. 

2) 

196. 

2( 

S0M2  EDUC  AFTER  HS 

1475 

175,858( 

3X) 

77. 

.2( 

1. 

3) 

14. 

6( 

231. 

.3( 

1. 

9) 

206. 

.9( 

GRADUATED  COLLEGE 

2356 

301, '•53( 

3X) 

80. 

.1( 

\. 

2) 

11. 

3( 

235 

.5( 

1, 

1) 

207, 

.3( 

UNKNOWN 

144 

14,413(12X) 

37 

.5( 

4. 

C) 

22, 

.6( 

193 

.6(12. 

2) 

166. 

K 

2 

.8) 

22. 2( 

2. 

0) 

2.9(  0.9) 

0. 

,9(  0.5) 

0.2( 

0. 

.2) 

0.0 

5 

.4) 

197. 0( 

4. 

7) 

212.7(21.3) 

196, 

.7(34.0) 

1 

.3) 

6.6( 

0. 

7) 

1.9(  0.5) 

0, 

.7(  0.3) 

0.0( 

0. 

.0) 

0.0 

4 

.7) 

207. 8( 

5. 

,5) 

220.2(10.3) 

198, 

.1(33.1) 

1 

.0) 

6.4( 

0, 

.8) 

1.3(  0.4) 

0, 

.4(  0.2) 

O.K 

0 

.1) 

0.0 

4 

.4) 

216. 8( 

5, 

.8) 

222.3(10.7) 

256. 

.6(24.8) 

0 

.9) 

4.0( 

0, 

.5) 

4.1(  0.5) 

0, 

.4(  0.2) 

0.0( 

0 

.0) 

0.0 

3 

.3) 

208. 2( 

6. 

.3) 

230. 6(  8.4) 

220 

.3(22.4) 

4 

.2) 

35. 3( 

4. 

.0) 

2.6(  1.3) 

2. 

.0(  1.5) 

0.0( 

0 

.0) 

0.0 

.4 

.5) 

182. 1( 

6. 

.0) 

180.9(28.6) 

166. 

.0(40.0) 

*  Average  response  method  proficiency  score 

**  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 
*****(****)    The  sample  size  is  insufficient  to  provide  reliable  estimates. 
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Table  16-88 


Weighted  Response  Percentages  and  Writing  Proficiency*  Means  with  Standard  Errors 
Grade  12,  by  Parental  Education,  for  Main  Writing  Samples 


PARENTS*  EDUCATION    (NAEP  ID:  PARED) 

N     WEIGHTED  N  (CV)**     KOT  HS            GRAD  HS  POST  HS  GRAD  COL  UNKNOWN  MISSING 

—TOTAL—                              6069       726,451(  2X)        7.5(  0.5)  Z^.Sv  0.8)  24. 2(  0.7)  41. 5(  1.1;  2.0(  0.2)  0.5 

208. 9(  3.0)  218. 9(  1.9)  226. d(  1.7)  231. 0(  KB)  182. A(  6.0) 


GENDER 

MALE                                     2934       3A6.235(  3Z)        6.6(  0.5)  25. 6(  1.1)  22. A(  0.9)  42. 8(  1.4)         2.6(  0.3)  0.4 

198. 1(  3.6)  208. 5(  2.5)  215. 6(  2.6)  218. 1(  1.8)  178. 8(  7.7) 

FEMALE                                  3135       380,216(  3Z)        8.3(  0.6)  24. 0(  0.9)  25. 9(  1.0)  40. 4(  1.5)         1.4(  0.3)  0.5 

216. 6(  4.5)  229. 0(  2.7)  235. 7C  2.3)  243. 4(  2.6)  188. 5(  9.3) 


RACE/ETHNICITY 


WHITE 

4178 

546,337( 

2Z) 

5. 

5( 

0. 

5) 

24. 

4( 

1. 

1) 

24 

9( 

0. 

8) 

44 

2( 

1 

2) 

1. 

0( 

0 

2) 

0 

.4 

220. 

2( 

4. 

3) 

225 

2( 

2. 

2) 

231 

3( 

1. 

9) 

235 

5( 

2 

1) 

i.q3. 

6(12. 

2) 

BUCK 

1095 

103, 257 ( 

4Z) 

9. 

9( 

1. 

6) 

29 

0( 

1. 

6) 

24 

9( 

1. 

7) 

33 

K 

2 

5) 

3 

K 

0 

8) 

0 

.6 

189. 

4( 

5. 

4) 

196 

2( 

4. 

7) 

206 

9( 

4. 

4) 

207 

3( 

3 

3) 

166. 

1(14 

5) 

HISP.\NIC 

612 

52,577( 

5Z) 

23. 

0( 

2. 

1) 

22 

6( 

9) 

21 

6( 

2. 

6) 

23 

K 

3 

0) 

9 

7( 

1 

0) 

0 

4 

197. 

0( 

4. 

7) 

20/ 

8( 

s! 

5) 

216 

8( 

5. 

8) 

208 

2( 

6 

3) 

182. 

K 

6 

0) 

OTHER 

184 

24, 280 ( 

8Z) 

8. 

8( 

2. 

3) 

19 

4( 

3. 

5) 

12 

5( 

2. 

9) 

56 

6( 

5 

1) 

2. 

7( 

1 

2) 

1 

.6 

209 

3(17. 

7) 

213 

9( 

9. 

9) 

230 

K 

9. 

5) 

229 

.6( 

6) 

174 

5(23 

.1) 

ARENTAL  EDUCATION 

LESS  THAN  HS 

541 

54, 507 ( 

7Z) 

100 

0( 

0 

0) 

0 

0( 

0. 

0) 

0 

.0( 

0. 

0) 

0 

.0( 

0 

.0) 

0 

0( 

0 

0) 

0 

.0 

208 

9( 

3 

0) 

GRADUATED  HS 

1524 

179,919( 

4Z) 

0 

0( 

0 

0) 

IOC 

0( 

0 

0) 

0 

0( 

0. 

0) 

0 

.0( 

0 

0) 

0 

0( 

0 

0) 

0 

.0 

218 

9( 

1. 

9) 

SOME  EDUC  AFTER  HS 

1475 

175,858( 

3Z) 

0 

0( 

0 

0) 

0 

.0( 

0. 

0) 

100 

.0( 

0 

0) 

0 

.0( 

0 

.0) 

0 

0( 

0 

0) 

0 

.0 

226 

.8( 

1 

7) 

GRADUATED  COLLEGE 

2356 

301,753( 

3Z) 

0 

.0( 

0 

0) 

0 

.0( 

0 

0) 

0 

.0( 

0 

0) 

100 

.0( 

0 

.0) 

0 

0( 

0 

.0) 

0 

.0 

231 

.0( 

3 

.8) 

UNKNOWN 

144 

14,413(12Z) 

0 

0( 

0 

0) 

0 

.0( 

0 

0) 

0 

.0( 

0 

0) 

0 

.0( 

0 

.0) 

100 

0( 

0 

0) 

0 

.0 

182 

4( 

6 

.0) 

*  Average  response  method  proficiency  scoce 

♦*  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 
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Table  16-89 


Weighted  Response  Percentages  and  Civics  Proficiency  Means  with  Standard  Errors 
Grade  4,  by  Gender,  for  Main  Focused-BIB  Civics  Samples 


GENDER  OF  SUBJECT    (NAEP  ID:  SEX) 


—  TOTAL 


GENDER 
MALE 

FEMALE 


RACE/ETHNICITY 
WHITE 

BLACK 

HISPANIC 

OTHER 


WEIGHTED  H  (CV)* 


MALE 


FEMALE 


MISSING 


1974 

375,529( 

IX) 

49. 1( 

1.3) 

50. 9( 

1.3) 

0.0 

214. 8( 

1.3) 

213. 3( 

1.1) 

978 

184,226( 

ZD 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0 

214. 8( 

1.3) 

996 

igi,303( 

3X) 

0.0( 

0.0) 

100. 0( 

0.0) 

C  0 

213. 3( 

1.1) 

1122 

262,686( 

2X) 

47. 8( 

1.5) 

52. 2( 

1.5) 

0.0 

220. 6( 

1.5) 

219. 5( 

1.2) 

360 

S8,616( 

5X) 

48. 0( 

3.6) 

52. 0( 

3.6) 

0.0 

390 

199. 7( 

2.8) 

196. 7( 

2.9) 

40.037( 

5X) 

56. 4( 

2.4) 

43. 6( 

2.4) 

0.0 

202. 1( 

2.4) 

196. 1( 

2.3) 

102 

14,ig0( 

8X) 

56. 5( 

4.9) 

43. 5( 

4.9) 

0.0 

213. 6( 

4.9) 

205. 4( 

7.5) 

PARENTAL  EDUCATION 
LESS  THAN  H.S. 

GRADUATED  H.S. 

SOME  EDUC  AFTER  H.S. 

GRADUATED  COLLEGE 

UNKNOWN 


116 

17,487(11X) 

39. 7( 

4.9) 

60. 3( 

4.9) 

0 

.0 

202. 9( 

4.5) 

210.6c 

4.7) 

269 

50,576(  7X) 

56. 2( 

3.1) 

43. e( 

3.1) 

0 

.0 

209. 2( 

2.5) 

213. 8( 

2.6) 

152 

31,922(10X) 

45. 3( 

4.6) 

54. 7( 

4.6) 

0 

.0 

712 

221. 9( 

4.2) 

221. 0( 

3.2) 

146,718<  5X) 

51. 4( 

2.1) 

48. 6( 

2.1) 

0 

.0 

222. 8( 

1.8) 

222. 3( 

1.7) 

716 

127,468(  4X) 

45. 4( 

2.3) 

54. 6( 

2.3) 

0 

.0 

207. 7( 

1.7) 

202. 5( 

2.2) 

*  CV  is  the  coefficient  of  variation  for  the  sum  of  the  wsights. 
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Table  16-90 


Weighted  Response  Percentages  and  Civics  Proficiency  Means    1th  Standard  Errors 
Grade  4,  by  Derived  Race/ethnicity,  for  Main  Focused-BIB  Civics  Samples 


DERIVED  RACE/ETHNICITY    (NAEP  ID:  DRACE) 


WEIGHTED  H  (CV)* 


WHITE 


BUCK 


HISPANIC 


ASIAN 


AKER  IND  UNCLASSIFIED 


MISSING 


—    TOTAL  — 

37S,529( 

IX) 

70. 0( 

0.9) 

15. 6( 

0.8) 

10. 7( 

0.6) 

2.0(  0.3) 

1.7( 

0.3) 

O.K  0.1) 

0.0 

220 . 0 ( 

1.0) 

198. 1( 

2.2) 

199. 5( 

1.9) 

210. 3(  8.3) 

209. 6( 

4.5) 

211.7(21.0) 

GENDER 

MALE 

18A,226( 

3X) 

68. 1( 

1.3> 

15. 3( 

1.0) 

12. 3( 

1.0) 

2.4(  O.S) 

2.0( 

0.4) 

0.0(  0.0) 

0.0 

220. 6( 

1.5) 

199. 7( 

2>d) 

202. 1( 

2.4) 

215. 7(  7.0) 

211. 0( 

6.9) 

FEMALE 

996 

191,303( 

3X) 

71. 7( 

1.5) 

15. 9( 

1.5) 

9.1( 

0.7) 

1.6(  0.4) 

1.5( 

0.4) 

0. 1(  0.1) 

0.0 

219. S( 

1.2) 

196. 7( 

2.9) 

196. 1( 

2.3) 

202.2(14.5) 

207. 9( 

6.9) 

211.7(21.0) 

RACE/ETHNICITY 

WHITE 

1122 

262, 686 ( 

2X) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0(  0.0) 

0.0( 

0.0) 

0.0(  0.0) 

0.0 

220. 0( 

1.0* 

BLACK 

360 

S8.616( 

5X) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0(  0.0) 

'*.0( 

0.0) 

0.0(  0.0) 

0.0 

198. 1( 

2.2) 

HISPANIC 

390 

irU,037( 

5X) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0(  0.0) 

0.0( 

0.0) 

0.0(  0.0) 

0.0 

199. 5( 

1.9) 

OTHER 

102 

1A»190( 

8X) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

51. 8(  6.4) 

^6.3( 

6.5) 

1.9(  1.4) 

0.0 

210. 3(  8.3) 

209. 6( 

4.5) 

211.7(21.0) 

PARENTAL  EDUCATION 

LESS  THAN  H.S. 

116 

17,A87(11Z) 

59. d( 

6.0) 

16. 7( 

3.7) 

17. 8( 

3.6) 

2.5(  2.5) 

2.2( 

1.3) 

1.0(  1.0) 

0.0 

21A.2( 

4.1) 

197. 1( 

8.8) 

202. 2( 

4.5) 

179. 4(****) 

184.3(19.1) 

200.7(16.3) 

GRADUATED  H.S. 

269 

50,576( 

7X) 

68. 2( 

2.8) 

18. 5( 

1.8) 

10. 4( 

1.7) 

1.2(  0.6) 

1.7( 

0.8) 

0.0(  0.0) 

0.0 

216. 3( 

2.0) 

199. 4( 

4.8) 

200. 3( 

4.4) 

212. 3(  4.9) 

200.6(12.0) 

SOME  EDUC  AFTER  H.S. 

152 

31.922(10Z) 

73. 2( 

3.6) 

15. 3( 

2.8) 

7.9( 

2.0) 

0.3(  0.3) 

3.3( 

1.6) 

0.0(  0.0) 

0.0 

27.6. 8( 

3.0) 

198. 3( 

6.5) 

215. 1( 

6.5) 

207. 5(****) 

224. 5( 

7.3) 

GRADUATED  COLLEGE 

712 

146,718( 

5X) 

74. 8( 

1.6) 

13. 4( 

1.2) 

7.8( 

1.1) 

2.4(  0.5) 

1.6( 

0.4) 

0.0(  0.0) 

0.0 

227. 3( 

1.3) 

204. 8( 

2.7) 

205. 9( 

3.2) 

227. 0(  6.5) 

221. 3( 

9.0) 

UNKNOWN 

715 

127,A68( 

66. 1( 

1.9) 

16. 3( 

1.6) 

13. 9( 

1.1) 

2.1(  0.5) 

1.5( 

0.4) 

O.K  0.1) 

D.O 

211. 1( 

1.6) 

192. 4( 

3.4) 

192. 4( 

2.6) 

192.6(10.4) 

196. 4( 

4.7) 

231. 3(****) 

*  CV  is  the  coefficient  o 
(****)  Standard  error  is 


variation  for  the  sum  of  the  neishts. 
eater  than  99.9. 
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Table  16-91 


Weighted  Response  Percentages  and  Civics  Proficiency  Means  with  Standard  Errors 
Grade  4,  by  Parental  Education,  for  Main  Focused-BIB  Civics  Samples 


PAREMTS^  EOUCATIOM    (XA2P  ID:  PARED) 

N        WSIGflTED  H  (CV)* 
—    TOTAl.  —  1985  37A.170(  IX) 


HOT  HS 

4.7(  0.5) 
207. 5(  3.5) 


GRAD  HS 

13. 5(  1.0) 
211. 2(  1.6) 


POST  HS 

8.5(  0.9) 
221. 4(  2.8) 


GRAD  COL 

39. 2(  1.6) 
222. 5(  1.3) 


UNKNOWN 

34. 1(  1.3) 
204. 8(  1.4) 


MISSING 
0.4 


GENDER 
HALE 

FEMALE 


971 
994 


183,131(  3Z) 
191,039(  3Z) 


3.8(  0.8) 

202. 9(  4.5) 

5.5(  0.8) 

210. 6(  4.7) 


15. 5(  1.5) 
209. 2(  2.5) 

11. 6(  1.1) 
213. 8(  2.6) 


7.0(  1.1) 

221. 9(  4.2) 

9.1(  l.X) 

221. 0(  3.2) 


41. 2(  2.0) 

222. 8(  1.8) 

37. 3(  2.2) 

222. 3(  1.7) 


31. 6(  1.4) 

207. 7(  1.7) 

36. 4(  2.0) 

202. 5f  2.2) 


0.6 
0.1 


RACE/ETHHICITY 


WHITE 

1120 

262,280(  21) 

4.0( 

0.7) 

13. 1( 

1.3) 

8.9( 

1.0) 

A1.8( 

1.9) 

3?..1( 

1.6) 

0.2 

214. 2( 

4.1) 

216. 3( 

2.0) 

226. 8( 

3.0) 

227. 3( 

1.3^ 

211. 1( 

1.6) 

BLACK 

353 

57,663(  6Z) 

5.1( 

1.2) 

16. 3( 

:.6) 

8.5( 

1.8) 

34. 1( 

2.t} 

36. 1( 

3.0) 

1.6 

197. 1( 

8.8) 

199. 4( 

4.8) 

108. 3( 

6.5) 

204. 8( 

2.7) 

192. 4( 

3.4) 

HISPANIC 

390 

40,037(  5Z) 

7.8( 

i.e) 

13. 1( 

2.1) 

6.3( 

1.6) 

28. 6( 

3.5) 

U.3( 

3.2) 

0.0 

202. 2( 

4.5) 

200. 3( 

4.4) 

315. 1( 

6.5) 

205. 9( 

3.2) 

192. 4( 

2.6) 

OTHER 

102 

14.190(  8Z) 

7.0( 

3.3) 

10. 2( 

3.3) 

8.1( 

3.6) 

41. 6( 

6.4) 

33. 1( 

4.9) 

0.0 

185.\(10.4) 

205. 4( 

8.2) 

223. 1( 

7.1) 

224. 7( 

5.0) 

195. 0( 

6.3) 

PARENTAL  EDUCATION 

Lf:.3S  THAN  B.S. 

116 

17,487(111) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0 

207. 5( 

3.5) 

GRADUATED  H.S. 

269 

50,576(  7Z) 

0.0( 

0.0) 

100. 0( 

0.0) 

o.:( 

0.0) 

0.0( 

0.0) 

O.sJi 

0.0) 

0.0 

211. 2( 

1.8) 

SOME  EDUC  AFTER  H.S. 

152 

31,922(101) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0 

221. 4( 

2.8) 

GRADUATED  COLLEGE 

712 

146,718(  5Z) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0 

222. 5( 

1.3) 

UNKNOWN 

716 

127»468(  4Z) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0 

204. 8( 

1.4) 

*  CV  i«  tht  co«mci«nt  of  v«ri«tion  for  th«  sum  of  th«  wcishts. 
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Table  16-92 

Weighted  Response  Percentages  and  Civics  Proficiency  Means  with  Standard  Errors 
GraJe  8,  by  Gender,  for  Main  Focused-BIB  Civics  Samples 


GENDER  OF  SUBJECT    (HAEP  ID:  SEX) 

K        WEIGSTED  N  (CV)*        KALE  FEMALE  MISSIH6 


-    TOTAL  — 

4486 

504, 400 ( 

IX) 

48 

8( 

0.8) 

51 

2( 

0 

8) 

0.0 

258 

7( 

1.1) 

260 

6( 

0 

9) 

EHDER 

MALE 

2155 

246,135( 

2X) 

100 

0( 

0.0) 

0 

0( 

0 

0) 

0.0 

258 

7( 

1.1) 

FEMALE 

2331 

258,265( 

2Z) 

0 

0( 

0.0) 

100 

0( 

0 

0) 

0.0 

260. 6(  0.9) 

RACE/ETHHICITY 


WHITE 

2708 

351,704( 

2X) 

48 

5( 

1. 

1) 

51 

5( 

1) 

0 

0 

265 

9( 

1. 

6) 

266 

7( 

1. 

3) 

BLACK 

805 

78,364( 

4Z) 

47 

7( 

1. 

5) 

52 

3( 

1. 

5) 

0 

1 

242 

2( 

2. 

^^ 

245 

0( 

1. 

9) 

HISPANIC 

736 

52,422t 

4X) 

51 

3( 

2 

1) 

48 

7( 

2. 

1) 

0 

0 

238 

1( 

2. 

4) 

243 

2( 

2. 

0) 

OTHER 

237 

21,909( 

7X) 

50 

7( 

4. 

0) 

49 

3( 

4. 

0) 

0 

0 

253 

3( 

5. 

0) 

258 

2( 

3. 

9) 

ARENTAL  EDUCATION 

LESS  THAN  H.S. 

410 

39,746( 

5X) 

44 

5( 

2. 

8) 

55 

5( 

2 

8) 

0 

0 

234 

.8( 

3 

2) 

240 

3( 

2 

8) 

GRADUATED  H.S. 

1229 

136, 360 ( 

4X) 

47 

.4( 

1 

8) 

52 

6( 

1 

6) 

0 

0 

250 

.9( 

1 

9) 

253 

9( 

1 

3) 

SOME  EDUC  AFTER  H.S. 

844 

97, 514 ( 

4X) 

46 

.6( 

1 

4) 

53 

4( 

1 

4) 

0 

0 

262 

.2( 

1 

9) 

264 

9( 

1 

5) 

GRADUATED  COLLEGE 

1630 

193, 521 ( 

4X) 

49 

.6( 

1 

4 

50 

4( 

1 

4) 

0 

0 

272 

.4( 

1 

6) 

272 

0( 

1 

4) 

UNKNOMIf 

357 

35, 603 ( 

6X) 

59 

.7( 

3 

6) 

40 

3( 

3 

6) 

0 

2 

235 

.8( 

3 

9) 

235 

.0( 

3 

2) 

*  CV  is  the  coafficlert  of  variation  for  the  sisn  of  thR  weights. 
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Table  16-93 

Weighted  Response  Percentages  and  Civics  Proficiency  Means  with  Standard  Errors 
Grade  8,  by  Derived  Race/ethnicity,  for  Main  Focused-BIB  Civics  Samples 


DERIVED  RACE/ETHNICITY    (NAF^  ID:  DRACE) 

N        WEIGHTED  N  (CV)* 


WHITE 


BUCK 


HISPANIC 


ASIAN 


AMER  IND  UHCLASSIFIED 


—    TOTAL  — 

4487 

504,472( 

IX) 

69. 

7( 

0. 

6) 

15. 

5( 

0.5) 

10. 

4< 

0. 

4) 

2. 

8< 

0. 

3) 

J  \ 

1 

1 . 

243 . 

1  .9) 

240 . 

R( 

I , 

262. 

7< 

4. 

1) 

GENDER 

6( 

«3  } 

KALE 

2156 

Z46, Z0o( 

2X) 

. 

4( 

n 
U . 

») 

1  c 

9  / 

n  7  ^ 

1  n 

Q( 

n 
u . 

7  \ 

2. 

0. 

Zb3 . 

9( 

1 . 

b; 

1\ 

9    7  \ 

*\ 

*♦  } 

262. 

5< 

6. 

FEMALE 

2331 

21) 

70 . 

K 

0 . 

Q  ^ 

y ) 

10 

o/ 

n  7  ^ 
U.  /  J 

Q 
9  . 

Q/ 

y^ 

n 

0) 

3 . 

o< 

0. 

■J  ^ 

^bb . 

7  / 
f\ 

1 . 

3  ) 

n  / 
v\ 

1    Q  ^ 

9AT 

9/ 

2 . 

262 

9( 

L  . 

RACE/EXali  ICIT  X 

o< 

0) 

WHITE 

2708 

351,704< 

21) 

100 

0( 

0 

0) 

0 

o< 

0.0) 

0 

o< 

0. 

0) 

0 

0 

266 

3( 

1 

2) 

BLACK 

806 

78,437( 

4Z) 

0 

0( 

0 

0) 

100 

o< 

0.0) 

0 

0( 

0. 

0) 

0 

0( 

0 

0) 

243 

6< 

HISPANIC 

736 

52,422( 

4Z) 

0 

0( 

0 

0) 

0 

o< 

0  ^, 

100 

0( 

0. 

0) 

0 

0( 

0 

'1) 

240 

6( 

1. 

7) 

OTHER 

237 

21,909( 

7Z) 

0 

0( 

0 

0) 

0 

o< 

0.0) 

C 

0( 

0 

P) 

64 

5( 

4 

5) 

262 

7( 

4 

1) 

PARENTAL  Ei)UCATION 

9( 

6) 

LESS  THAN  H.S. 

410 

39,746< 

5Z) 

56 

5< 

3 

5) 

11 

3( 

1.7) 

28 

2< 

2 

9) 

1 

0 

243 

.4< 

2 

8) 

231 

5( 

4.5) 

228 

.9( 

3 

3) 

249 

4<20 

1) 

GRADUATfcO  H.S. 

1228 

136.360< 

4Z) 

67 

.7< 

J 

5) 

17 

.8( 

1.3) 

10 

5( 

1 

0) 

1 

7< 

0 

3) 

258 

.0< 

1 

6) 

238 

.9( 

2.6) 

240 

.7( 

2 

5) 

247 

2< 

6 

8) 

SOME  EDUC  AFTER  H.S. 

844 

97,514( 

4Z) 

73 

.6< 

1 

8) 

15 

.6( 

1.5) 

8 

.7( 

0 

7) 

1 

.2< 

0 

4) 

267 

.5< 

1 

3) 

252 

.7( 

2.9) 

251 

.3( 

3 

1) 

270 

.2< 

9 

1) 

GRADUATED  COLLEGE 

1630 

193,521< 

4Z) 

75 

.6( 

1 

.2) 

14 

.K 

0.9) 

5 

.6< 

0 

6) 

3 

.9< 

j 

5) 

277 

.7( 

1 

.4) 

250 

.7< 

2.9) 

251 

.8( 

3 

5) 

273 

.8< 

5 

.2) 

UNKNa.N 

358 

35,677< 

6Z) 

49 

.8( 

3 

.5) 

19 

.2( 

2.0) 

20 

.6( 

2 

1) 

6 

.5f 

2 

.6) 

244 

.K 

3 

.6) 

223 

■  K 

4.6) 

229 

.K 

4 

4) 

242 

.5(11 

.6) 

1.4(  0.2) 
242. 3(  4.4) 


1.7<  0.3) 
239. 8(  >./.) 

1.2<  0.2) 
245. 7<  4.4) 


0.0(  0.0) 

0.0<  0.0) 

0.0(  0.0) 

33. 4(  4.5) 

242. 3<  4.4) 


2.0<  0.7) 
231.4(11.1) 

r.3(  0.4) 
25:. 5<  5.4) 

J.8<  0.3) 
248. 3<  7.7) 

0.6<  0.2) 
256. 6<  6.8) 

3.6<  1.5) 
207.2(10.3) 


O.K  0.0) 
254.5(18.5) 


0.2(  0.1) 
246.6(19.5) 

O.U(  0.0) 
29?.. 6(****) 


0.0(  0.0) 

0.0(  0.0) 

0.0(  0.0) 

2.1(  1.0) 
254.5(18.5) 

0.0(  0.0) 
0.0(  0.0) 

O.K  0.1) 
283.7(16.2) 

O.K  0.1) 
269.2(10.8) 

0.3(  0.3) 
201.1(15.5) 


MISSING 
0.0 

0.0 
0.0 

0.0 
0.0 
0.0 
0.0 

0.0 
0.0 
0.0 
0.0 
0.0 


*  CV  is  the  coefficient  of  variation  for  the  sura  of  the  woishts. 
<****)  Standard  error  is  greater  than  99.9. 
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Table  lb  94 


Weighted  Response  Percentages  and  Civics  Proficiency  Means  with  Standard  Errors 
Grade  8,  by  Parental  Education,  for  Main  Focused-BIB  C5vics  Samples 


PARENTS'  EDUCATIOK    (NAEP  ID:  PARED) 

H        WEIGHTED  N  (CV)*         NOT  HS  (^AD  HS  POST  HS  GRAD  COL  UNKNOWN  MISSING 

—    TOTAL  —                            ^A70          502,817(  IX)           7.9(  0.«)  27. 1(  1.0)  19. A(  0.7)  38. 5(  1.4)         7.1(  O.A)  0.3 

237. 8(  2.2)  252. 5(  1.3)  263. 7(  1.3)  272. 2(  1.1)  235. A(  2.7) 


GENDER 

MALE                                      2145          245,061(  21)           7.2(  0.6)  26. 4(  1.3)  18. 6(  0.9)  39. 2(  1.8)  8.7(  0.7)  0.5 

r^t.3(  3.2)  250. 9(  1.9)  262. 2(  1.9)  272. 4(  1.6)  235. 7(  3.9) 

FEMALE                                   2325          257,756(  21)           8.6(  0.6)  27. 8(  1.3)  20. 2(  0.9)  37. 8(  1.4)  5.6(  0.6)  0.2 

240. 3(  2.8)  253. 9(  1.3)  264. 9(  1.5)  272. 0(  1.4)  235. 0(  3.2) 


RACE/ETHKICITY 
WHITE 

*.  XK 

HISPANIC 

OTHER 

PARENTAL  EDUCATION 
LESS  THAf»  H.S. 

GRADUATED  H.S. 

SOME  EDU'.  AFTER  H.S. 

GRADUATED  COLLEGE 

UNKNOWN 


2700  350,678( 

801  77,984( 

732  52,246( 

237  21,909( 

410  39,746( 

1228  136,360( 

844  97,514( 

1630  193, 521 ( 

358  35,677( 


21) 

6.4( 

0.6) 

243. 4( 

2.8) 

hi) 

5.8( 

1.0) 

231. 5( 

4.5) 

4X) 

21. 5( 

1.9) 

228. 9( 

3.3) 

71) 

7.2( 

1.6) 

240.2(10.4) 

52)        100. 0(  0.0) 
237. 8(  2.2) 
42)  0.0(  0.0) 

4Z)  0.0(  0.0) 

4Z)  0.0(  0.0) 

62)  0.0(  0.0) 


26. 3(  1.2)  20. 5( 

258. 0(  1.6)  267. 5( 

31. 1(  2.0)  19. 5( 

238. 9(  2.6)  252. 7( 

27. 4(  2.2)  16. 2( 

240. 7(  2.5)  251. 3( 

25. 1(  3.3)  9.4( 

250. 3(  4.6)  262. 2( 


0.0(  0.0)  0.0( 

100. 0(  0.0)  0.0( 

252. 5(  1.3) 

0.0(  0.0)  100. 0( 
263. 7( 

0.0(  0.0)  0.0( 

0.0(  0.0)  0.0( 


0.9)  41. 7(  1.8) 

1.3)  277. 7(  1.4) 

1.8)  34. 9(  1.8) 

2.9)  250. 7(  2.9) 
1.5)  20. 9(  2.0) 
3.1)  251. 8(  3.5) 
2.5)  41. 3(  4.7) 
6.9)  271. 3(  4.7) 


0.0)  0.0(  0.0) 

0.0)  0.0(  0.0) 

0.0)  0.0(  0.0) 
1.3) 

0.0)  100. 0(  0.0) 
272. 2(  1.1) 

O.C)  0.0(  0.0) 


5.1(  0.5)  0.3 

244. 1(  3.6) 

8.8(  1.0)  C.6 

223. 1(  4.6) 

14. 1(  1.4)  0.3 

229. 1(  4.4) 

17. 0(  4.6)  0.0 
228.9(11.3) 

0.0(  0.0)  0.0 

0.0(  0.0)  0.0 

0.0(  0.0)  0.0 

0.0(  0.0)  0.0 

100. 0(  0.0)  0.0 


235. 4(  2.7) 


^  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 
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Table  16-95 


Weighted  Response  Percentages  and  Civics  Proficiency  Means  with  Standard  Errors 
Grade  12,  by  Sender,  for  Main  Focused-BIB  Civics  Samples 


GEHDER  OF  SUBJECT 


(NAEP  ID:  SEX) 
H 


WEIGHTED  N  (CV)* 


MALE 


FEMALE 


MISSING 


—    TOTAL  — 

A275 

525, 727 ( 

IX) 

A8. 

A( 

1. 

6) 

51. 

6( 

1. 

6) 

0. 

0 

298. 

6( 

1. 

6) 

294. 

K 

1. 

1) 

GENDER 

MALE 

2049 

254, 193 ( 

3X) 

100. 

0( 

0. 

0) 

0. 

0( 

0. 

0) 

0. 

0 

298. 

6( 

1. 

6) 

FEMALE 

2226 

271,533( 

AX) 

0. 

0( 

C . 

0) 

100. 

0( 

0. 

0) 

0. 

0 

294. 

K 

1. 

1) 

RACE/ETHNICITY 

WHITE 

2960 

397,203( 

2X) 

A7 

8( 

1 

8) 

52 

2( 

1 

8) 

0. 

0 

305 

0( 

1 

7) 

299 

K 

1 

2) 

BLACK 

678 

62,248( 

5X) 

A7 

5( 

2 

9) 

52 

5( 

2 

9) 

0. 

G 

278 

K 

3 

1) 

269 

9( 

2 

3) 

HISPANIC 

487 

A3,9Ci( 

AX) 

52 

K 

2 

3) 

47 

9( 

2 

8) 

0. 

0 

277 

6( 

3 

8) 

281 

0( 

2 

9) 

OTHER 

ISO 

22.370(101) 

53 

9( 

3 

8) 

46 

K 

3 

8) 

0. 

0 

289 

A( 

7 

2) 

297 

5( 

o 

0) 

PARENTAL  EDUCATION 


LESS  THAN  H.S. 

374 

38,467( 

7X) 

44 

9( 

3 

4) 

55 

K 

3 

4) 

0 

0 

272 

4( 

3 

9) 

273 

4( 

3 

5) 

GRADUATED  H.S. 

1026 

125,616( 

4X) 

47 

2( 

2 

4) 

52 

8( 

2 

4) 

0 

0 

286 

3( 

2 

4) 

283 

8( 

1 

6) 

SOME  EDUC  AFTER  H.S. 

1056 

125. 510 ( 

5X) 

46 

7( 

1 

9) 

53 

3( 

1 

9) 

0 

0 

300 

.9( 

2 

5) 

296 

6( 

1 

5) 

GRADUATED  COLLEGE 

1708 

222, 853 ( 

4X) 

49 

4( 

1 

8) 

50 

6( 

1 

8) 

0 

0 

311 

2( 

1 

9) 

303 

8( 

1 

6) 

UNKNOWN 

97 

11.246(13X) 

67 

5( 

5 

1) 

32 

5( 

5 

1) 

0 

0 

266 

8( 

7 

.9) 

263 

7( 

7 

7) 

*  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weishts. 
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Table  16-96 


Weishted  Response  Percentages  and  Civics  Proficiency  Means  with  Standard  Errors 
Grade  12,  by  Derived  Race/ethnicity,  for  Main  F^cused-BIB  Civics  Samples 


DERIVED  RACE/ETHNICITY    (NAEP  ID:  DRACE) 


WEIGHTED  N  (CV)* 


WHITE 


BLACK 


HISPARIC 


ASIAN 


AMER  IKD  UNCLASSIFIED 


MISSING 


—    TOTAL  — 

A275 

525,727(  IZ) 

75. 6( 

0.8) 

11. 8( 

0.6) 

8.4( 

O.A^ 

O.K 

0.4^ 

1.0( 

0.3) 

O.K  0.1) 

0.0 

301. &( 

1.2) 

273. 8( 

1.9) 

279. 2( 

2.3/ 

302. :>( 

5.5) 

274. 2( 

8.6) 

218.2(22.5) 

GENDER 

MALE 

2049 

25A,193(  32) 

7A.6( 

1.1) 

11. 6( 

0.8) 

9.0( 

0.6) 

3.:^( 

0.5) 

1.2i 

0.4' 

0.3(  0.2) 

0.0 

305. 0( 

1.7) 

278. i< 

3.1) 

277. 6( 

3.U) 

302. 2( 

7.6) 

271.1(10.1) 

?,18. 2(22.5) 

FEMALE 

2226 

271,533(  AZ) 

76. 

1.1) 

12. 0( 

0.8) 

7.7( 

0.3) 

3.0( 

0.5) 

0.8( 

J.0(  0.0) 

0.0 

299. 1( 

1.2) 

Z69.9( 

2.3) 

281. 0( 

2.9) 

302. 9( 

6.3) 

278. 4< 

9.9) 

RACE/ETHNICITY 

WHITE 

2960 

397, 203 (  2Z) 

100. 0( 

G.O) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0)' 

0.0( 

0.0) 

0.0(  0.0) 

0.0 

301. 9( 

1.2) 

BUCK 

678 

62,2A8(  5Z) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0(  0.0) 

0.0 

273. 8( 

1.9) 

HISPANIC 

A87 

A3,906(  AZ) 

3.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0(  0.0) 

0.0 

279. 2( 

2.3) 

OTHER 

150 

22,370(10Z) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

73. 0( 

7.1) 

23. 9( 

6.9) 

3.1(  2.0) 

0.0 

302. 5( 

5.5) 

274. 2( 

8.6) 

218.2(22.5) 

PARENTAL  EDUCATION 


LESS  THAN  H.S. 

374 

38,467( 

7Z) 

55. 2( 

3.7) 

14. 5( 

2.6) 

26. 8( 

2.4) 

2.8(  1.0) 

0,7(  0.6) 

0.0( 

0.0) 

0.0 

277. 4( 

3.4) 

261. 0( 

5.0) 

270. 8( 

4.1) 

276.8(13.1) 

238.6(24.5) 

GR/BUATED  H.S. 

1026 

125,616( 

4Z) 

74. 4( 

1.9) 

14. 7( 

1.4) 

7.9( 

0.8) 

1.6(  0.7) 

1.4(  0.6) 

O.K 

0.1) 

0.0 

290. 1( 

1.7) 

265. 5( 

2.8) 

276. 6( 

3.9) 

288.5(19.4) 

263.5(15.0) 

270.6(24.3) 

SOME  EDUC  AFTER  H.S. 

105*^ 

125,510( 

5Z) 

77. 3( 

1.6) 

13. 0( 

1.1) 

7.3( 

0.8) 

1.3(  0.5) 

1.0(  0.3) 

0.0( 

0.0) 

0.0 

303. 4( 

1.6) 

279. 8( 

2.7) 

279. 9( 

5.3) 

318.4(14.6) 

285.3(12.2) 

GRi\»  JATED  COLLEGE 

1708 

222, 853 ( 

4Z) 

80. 5( 

1.1) 

8.5( 

0.7) 

5.3( 

0.5) 

4.9(  0.7) 

0.7(  0.4) 

O.K 

0.1) 

0.0 

311. 2( 

1.6) 

284. 7( 

2.9) 

291. 4( 

4.1) 

308. 0(  5.5) 

280. 5(  6.3) 

203. 4( 

9.7) 

UNKNOWN 

97 

11,246(13Z) 

45. 6( 

6.8) 

21. 1( 

4.2; 

24. 5( 

4.9) 

5.6(  2.0) 

3.1(  2.2) 

0.0( 

0.0) 

0.0 

275.8(10.2) 

241. 2( 

6.3) 

265. 7( 

8.5) 

266.7(32.2) 

285.0(16.7) 

*  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 


ERIC 

hnimiiiiTirn'nma 


4S5 


455 


Table  16-97 


Weighted  Response  Percentages  and  Civics  Proficiency  Means  with  Standard  Errors 
Grade  12,  by  Parental  Education,  for  Main  Focused-BIB  Civics  Samples 


PAREHTS'  EijJCATlOH    (HAEP  ID:  PARED) 

K        WEIGHTED  H  (CV)* 
—    TOTAL  —  A261  523.692(  U) 


HOT  HS 

7.3(  0.5) 
273. 0(  2.5) 


GRAD  HS 

24. 0(  1.1) 
285. 0(  1.6) 


POST  HS 

24. 0(  1.1) 
298. 6(  1.4) 


GRAD  COL 

42. 6(  1.6) 
307. 4(  1.5) 


UNKKOWN 

2.1(  0.3) 
265. 8(  5.7) 


MISSING 
0.4 


GEHDER 
MALE 

FEMALE 


2041 
2220 


2S2,d71(  3Z) 
270,821(  4Z) 


6.8(  0.6) 

272. 4(  3.9) 

7.8(  0,7) 

273, 4(  3,5) 


23, 5(  1.4) 

286. 3(  2.4) 

24. 5(  1,1) 

283. 8(  1,6) 


23. 2(  1,A> 

300. 9(  2,5) 

24. 7(  1,1) 

:j6.6(  1,5) 


43, 5(  2,0) 

311, 2(  1,9) 

41. 6(  1.7) 

303. 8(  1,6) 


3.0(  0,5) 

266. 8(  7,9) 

1,3(  0,2) 

263, 7(  7.7) 


0.5 
0,3 


RACE/ETHNICITY 


WHITE 

2952 

396,245(  21) 

5.4( 

0.6) 

23, 6( 

1.3) 

24. 5( 

1.3) 

45. 3( 

1.8) 

1.3(  0.3) 

0.2 

277. 4( 

3.4) 

290, 1( 

1.7) 

303, 4( 

1,6) 

311. 2( 

1.6) 

275.8(10,2) 

BLACK 

674 

61,628(  5Z) 

9,0( 

1.6) 

29, 9( 

2.0) 

26, 4( 

2.0) 

30. 9( 

2.4) 

3.9(  0,8) 

1.0 

261, 0( 

5,0) 

265. 5( 

2.8) 

279. 8( 

2,7) 

284, 7( 

2.9) 

241. 2(  6,3) 

HISPANIC 

487 

43,906(  4Z) 

23. 5( 

2,1) 

22. 6( 

1,9) 

20. 9( 

2,2) 

26. 7( 

2.5) 

6.3(  1,3) 

0.0 

270. 8( 

4.1) 

276. 6( 

3,9) 

279. 9( 

5.3) 

291. 4( 

4.1) 

265, 7(  8,5) 

OTHER 

148 

21,913(102) 

6.2( 

2.0) 

17, 5( 

4.3) 

13. 7( 

2,9) 

58. 1( 

6.6) 

4,5(  1.5) 

2.0 

268.9(11,7) 

276,5(10.2) 

304,0(11,1) 

302. 8( 

5,8) 

273,2(16.5) 

PARENTAL  EDUCATION 

LESS  THAN  H.S, 

374 

38,467(  7X) 

100. 0( 

0,0) 

0,0( 

0,0) 

0.0( 

0,0) 

0,0( 

0.0) 

0,0(  0.0) 

0,0 

273. 0( 

2.5) 

GRADUATED  H.S. 

1026 

125.616(  4Z) 

0,0( 

0.0) 

100, 0( 

0.0) 

0,0( 

0.0) 

0.0( 

0,0) 

0.0(  0,0) 

0.0 

2^»S.0( 

1.6) 

SOME  EDUC  AFTER  H.S, 

1056 

125,510(  5Z) 

0.0( 

0,0) 

O.C 

0,0) 

100. 0( 

0,0) 

0,0( 

0.0) 

0.0(  0,0) 

0,0 

298. 6( 

1.4) 

GRADUATED  COLLEGE 

1708 

222,853(  4Z) 

0.0( 

0,0) 

0,0( 

0,0) 

0.0( 

0,0) 

100. 0( 

0,0) 

0.0(  0.0) 

0,0 

307. 4( 

1.5) 

UNKNOWN 

87 

11,246(131) 

0.0( 

0.0) 

0,0( 

0,0) 

0,0( 

0,0) 

0.0( 

0.0) 

100. 0(  0,0) 

0.0 

265. 8(  5.7) 


ERIC 

umnmnimu 


*  CV  is  the  cosfficient  of  variation  for    .i«  sua  of  the  weishts. 
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Table  16-98 


Weighted  Response  Percentages  and  U.S.  History  Proficient.    Aeais  with  Standard  Errors 
Grade  4,  by  Gender,  for  Main  U.S,  History  Samples 


G£in)ER  OF  SUBJECT    (HAEP  ID:  SEX) 


N 


WEIGHTED  H  (CV)* 


KALE 


FEMALE 


*  CV      tho  coo£fici«nt  of  variation  for  th«  sum  of  tho  weishts. 


KISSINO 


TOTAL  — 

3950 

751,664(  IZ) 

50. 7( 

0.9) 

49. 3( 

0.9) 

0.0 

222. 9( 

1.2) 

218. 2( 

1.0) 

GENDER 

MALE 

2026 

381,243(  2X) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0 

222. 9( 

1.2) 

FEMALE 

1924 

370,420(  2X) 

0.0( 

0.0) 

100. 0( 

0.0) 

0«0 

ai8.Z( 

1.0) 

RACE/ETHNICITY 

WHITE 

2258 

531,472(  IZ) 

49  9( 

1.2) 

50. 1( 

1.2) 

0.0 

230. 8( 

1.3) 

224. 4( 

1.2) 

BLACK 

682 

108,783(  3Z) 

47. 8( 

2.1) 

52. 2( 

2.1) 

0.0 

200. 1( 

2.6) 

198. 9( 

2.3) 

HISPAHXC 

811 

82,452(  3Z) 

55. 4( 

1.9) 

44. 6( 

1.9) 

0.0 

204. 5( 

2.1) 

200. 5( 

2.4) 

OTHER 

199 

2e.956(  8Z) 

63. 4( 

3.1) 

36. ec 

3.1) 

0.0 

219. 7( 

4  9) 

227. 2( 

4.8) 

PARENTAL  EDUCATION 

LESS  THAK  H.S. 

240 

38.427(10Z) 

47. 7< 

3.2) 

52. 3( 

3.2) 

0.0 

206. 7( 

3.1) 

199. 0( 

2.6) 

GRADUATED  H.S. 

531 

104,142(  6Z) 

49. 8( 

2.5) 

50. 2( 

2.5) 

0.0 

215. 6( 

2.1) 

212. 6( 

2.0) 

SOME  EDUC  AFTER  H.S. 

314 

60,114(  6Z) 

52. 9( 

3.2) 

47. 1( 

3.2) 

0.0 

230. 0( 

3.7) 

225. 8( 

2.4) 

GRADUATED  COLLEGE 

1-1X6 

2d5.166(  4Z) 

53. 6( 

1.5) 

46. 4( 

1.5) 

0.0 

233. 3( 

1.8) 

229. 1( 

1.6) 

U.KKNOWN 

1432 

261.658(  3Z) 

48. 0( 

1.7) 

52. 0( 

1.7) 

0.0 

213. >( 

1.0 

211. 2( 

1.3) 

ERIC 

hnimiiimrn'nma 


QQ 
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Table  16-99 


Weighted  Response  Percentages  and  U.S.  History  Proficiency  Means  with  Standard  Errors 
Grade  4,  by  Derived  Race/ethnicity,  for  Main  U.S.  History  Samples 


DERIVED  EACE/ETHNICITY    (HAEP  ID:  DRACE) 


WEICTTED  K  (CV)* 


WHITE 


BUCK 


HISPANIC 


ASIAN 


AMER  IHD  UNCLASSIFIED 


MISSING 


—   TOTAL  — 

3950 

751,664( 

IX) 

70. 

7( 

0. 

6) 

14. 

5( 

0. 

4) 

11. 

0( 

0 

4) 

2 

0( 

0. 

2) 

1, 

8( 

0. 

2) 

0. 

1( 

0. 

0) 

0.0 

227 

5( 

1. 

0) 

199. 

5( 

1. 

9) 

202. 

7( 

1 

7) 

231 

3( 

5. 

7) 

212. 

7( 

4. 

7) 

208 

6(10. 

7) 

GENDER 

5) 

K 

1) 

MALE 

2026 

381,243( 

2X) 

69 

6( 

0. 

9) 

13 

6( 

0. 

7) 

12 

0( 

0 

6) 

2 

K 

0. 

3) 

2. 

7( 

0 

0 

0. 

0.0 

230 

8( 

1 

3) 

20C 

1( 

2. 

6) 

204. 

5( 

2 

1) 

226 

0( 

7. 

6) 

214. 

9( 

6 

5) 

214 

5( 

7, 

3) 

FEMALE 

1924 

370,420( 

2X) 

71 

9( 

0. 

9) 

15 

3< 

0. 

7) 

9 

9( 

0 

4) 

2 

0( 

0. 

3) 

0. 

8( 

0 

2) 

0. 

1( 

0. 

1) 

0.0 

224 

4( 

1 

2) 

198 

9( 

2. 

3) 

200 

5( 

2 

4) 

237 

0( 

6. 

0) 

205 

7( 

8 

2) 

202 

HACE/ETHNICITY 

0) 

0( 

0) 

WHITE 

2258 

531,472( 

IX) 

100 

0( 

0 

0) 

0 

0( 

0. 

0) 

0 

0( 

0 

.0) 

0 

0( 

0. 

0) 

0. 

0( 

0 

0 

0. 

0.0 

227 

.5( 

1 

0) 

BLACK 

682 

108,783( 

3X) 

0 

0( 

0 

0) 

100 

0( 

0. 

0) 

0 

0( 

0 

.0) 

0 

0( 

0. 

0) 

0 

0( 

0 

0) 

0 

0( 

0. 

0? 

0.0 

199 

5( 

1 

9) 

HISPANIC 

811 

82,452( 

3X) 

0 

.0( 

0 

.0) 

0 

0( 

0 

0) 

100 

0( 

0 

.0) 

0 

0( 

0. 

0) 

0 

0( 

") 

0 

0( 

0. 

0) 

0.0 

202 

7( 

1 

.7) 

OTHER 

199 

28,956( 

8X) 

0 

.0( 

0 

.0) 

0 

.0( 

0 

0) 

0 

0( 

0 

.0) 

52 

.8( 

4. 

8) 

45 

8( 

5 

0) 

1 

.4( 

1 

0) 

0.0 

231 

.3( 

5. 

7) 

212 

7( 

4 

.7) 

208 

.6(10 

7) 

PARENTAL  EDUCATION 
LESS  THAN  H.S. 

GRADUATED  H.S. 

SOME  EDUC  AFTER  H.S. 

GRADUATED  COLLEGE 

UNKNOWN 


240 

38,427(10X) 

59. 0( 

3.3) 

19. 3( 

3.2) 

17 

9( 

2.6) 

2.3(  1.0) 

1.6(  0.8) 

O.C(  0.0) 

0.0 

211. 4( 

3.2) 

184. 4( 

5.0) 

196 

3( 

3.5) 

197. 0(  9.6) 

183.8(16.9) 

531 

104,142(  6X) 

70. 2( 

2.0) 

ld.5( 

1.7) 

8 

2( 

0.9) 

0.8(  0.3) 

2.3(  0.7) 

0.0(  0.0) 

0.0 

220. 5( 

1.7) 

197. 7( 

2.4) 

200 

7( 

4.6) 

204.6(23.3) 

201.3(15.1) 

0.0(  0.0) 

314 

60,114(  6X) 

75. 0( 

2.7) 

11. 9( 

1.9) 

10 

9( 

1.6) 

0.4(  0.3) 

1.8(  0.6) 

0.0 

234. 9( 

2.4) 

205. 2( 

5.5) 

204 

8( 

6.0) 

252.7(14.8) 

225. 2(  9.0) 

1416 

285,1C5(  4X) 

73. 9( 

1.1) 

13. 1( 

0.9) 

8 

8( 

0.7) 

2.6(  0.4) 

1.4(  0.4) 

O.K  0.1) 

0.0 

237. 7( 

1.4) 

206. 8( 

2.8) 

211 

3( 

2.3) 

247. 9(  5.7) 

223. 2(  8.0) 

208.6(10.7) 

1432 

261,658(  3X) 

68. 5( 

1.6) 

13. 9( 

1.3) 

13 

3( 

0.9) 

2.2(  0.6) 

2.0(  0.4) 

0.0(  0.0) 

0.0 

218. 7( 

1.3) 

195. 2( 

2.9) 

197 

9( 

2.3) 

218. 2(  5.4) 

210. 6(  ^^.3) 

ERIC 


*  CV  is  th«  cotfficitnt  of  variation  for  th«  sua  of  the  woishts. 

(****}  standard  arror  ia  creator  than  99.9.  O 
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Table  16-100 


Weighted  Response  Perc  mtages  and  U.S.  History  Proficiency  Means  with  Standard  Errors 
Grade  4,  by  Parental  Education,  for  Main  U.S.  History  Samples 


PARENTS'  EDUCATION    (KAEF  ID:  FARED) 

N        WEIGHTED  N  (CV)*         NOT  HS  C3^AD  HS  POST  HS  GRAD  COL  UNKNOWN  MISSING 

—    TOTAL  —                           3933          749,507(  IX)          5,1(  0.5)  13. 9(  0.8)  8.0(  0.5)  38. 0(  1.3)  3*.9(  0.9)  0.3 

202. 7(  2.2)  214. 1(  1.3)  228. 0(  2.4)  231. 4(  1.5)  212. 5(  1.0) 


GENDER 

MALE                                    2017          380,328(  2X)          4.8(  0.5)  13. 6(  0.9)  8.4(  0.6)  40.?.(  1.6)  33. 0(  1.3)  0.2 

206. 7(  3.1)  215. 6(  2.1)  230. 0(  3.7)  233. 3<  1.8)  213. 9(  1.4) 

FEMALE                                 1916          369.179(22)          5.4(0.7)  14.2(1.1)  7.7(0.7)  35.9(1.6)  36.9(13)  0.3 

199. 0(  2.6)  212. 6(  2.0)  225. 8(  2.4)  229. 1(  1.6)  211. 2(  1.3) 


RACE/ETHNICITY 


WHITE 

2256 

530,946( 

U) 

4.3( 

0.5) 

211. 4( 

3.2) 

BLACK 

673 

107,607( 

31) 

6.9( 

1.5) 

184. 4( 

5.0) 

HISPANIC 

805 

81,998( 

3X) 

8.4( 

1.1) 

196. 3( 

3.5) 

OTHER 

199 

28,956( 

8X) 

5.1( 

1.5) 

191. 5( 

9.0) 

ARENTAL  EDUCATION 

LESS  THAN  H.S. 

240 

38,427(10Z) 

100. 0( 

0.0) 

202. 7( 

2.2) 

GRADUATED  H.S. 

531 

104,142( 

6X) 

o.c. 

0.0) 

SOME  EDUC  AFTER  H.S. 

314 

60,114( 

6X) 

0.0( 

0.0) 

GRADUATED  COLLEGE 

1416 

285,166( 

4X) 

0.0( 

0.0) 

UNWJOWN 

1432 

261,658( 

3X) 

0.0( 

0.0) 

13. 8( 

1.0) 

8.5( 

0.6) 

39. 7( 

1.5) 

33. 8( 

1.1) 

0.1 

220. 5( 

1.7) 

234. 9( 

2.4) 

237. 7( 

1.4) 

218. 7( 

1.3) 

17. 9( 

1.5) 

6.7( 

1.1) 

34. 7( 

2.5) 

33. 9( 

3.0) 

1.1 

197. 7( 

2.4) 

205. 2( 

5.5) 

206. 8( 

2.8) 

195. 2( 

2.9) 

10. 4( 

1.1) 

8.0( 

1.1) 

30. 7( 

2.5) 

42, 5( 

2.1) 

0.6 

200. 7( 

4.6) 

204. 8( 

6.0) 

211. 3( 

2.3) 

197. 9( 

2.3) 

11. 2( 

3.0) 

4.5( 

1.5) 

41. 1( 

4.8) 

38. 1( 

4.9) 

0.0 

202.2(11.7) 

230. 5( 

7.8) 

238. 2( 

i,5) 

214. 7( 

4.7) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

CO) 

0.0( 

0.0) 

O.'' 

214. 1( 

1.3) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0 

228. 0( 

2.4) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0 

231. 4( 

1.5) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0 

212. 5( 

1.0) 

*  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 
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Table  16-101 


Weighted  Response  Percentages  and  U.S.  History  Proficiency  Means  with  Standard  Errors 
Grade  8,  by  Gender,  for  Main  U.S.  History  Samples 


GENDER  OF  SUBJECT    (NAEP  ID:  SEX) 


N 


WEIGHTED  K  (CV)* 


MALE 


FEMALE 


MISSIHG 


—    TOTAL  — 

6460 

731,581( 

U) 

49. 5( 

o.e) 

50. 5( 

0.8) 

0.0 

2t;s.2( 

1.0) 

261. 6( 

0.8) 

GENDER 

MALE 

3156 

362,158( 

2Z) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.1 

266. 2( 

1.0) 

FEHALE 

3304 

36g,423( 

2X) 

v.0( 

0.0) 

100. 0( 

0.0) 

0.0 

261. 6( 

0.8) 

RACE/ETHNICITY 

WHITE 

4006 

521.5.;d( 

IX) 

50. 1( 

0.9) 

49. g( 

0.9) 

0.0 

272. 7( 

1.2) 

268. 1( 

0.9) 

BLACK 

1094 

106,016( 

ZD 

44. g( 

1.6) 

55. 1( 

1.6) 

0.1 

248. 7( 

2.1) 

243. 8( 

1.7) 

HISPANIC 

1068 

77,623; 

3X) 

53. 7( 

1.5) 

46. 3( 

1.5) 

0.0 

245. 6( 

2.4) 

242. 9( 

2.0) 

OTHER 

292 

26,3g6( 

7X) 

44. 2( 

4.0) 

55. 8( 

4.0) 

0.4 

264. 8( 

4.5) 

263. 1( 

3.0) 

P/iREIITAL  EDUCATION 

LESS  THAN  H.S. 

587 

58,381( 

4X) 

38. 2( 

2.1) 

61. 8( 

2.1) 

0.0 

245. 0( 

2.7) 

244. 8( 

2.1) 

GRADUATED  H.S. 

1701 

ig6,123( 

3X) 

50. 0( 

1.4) 

50. 0( 

1.4) 

0.1 

257. 4( 

1.9) 

254. 7( 

1.2) 

SOME  EDUC  AFTER  H.S. 

1195 

135.031( 

3X) 

50. 1( 

1.6) 

49. 9( 

1.5) 

0.0 

272. 6( 

1.9) 

265. 6( 

1.5) 

GRADUATED  COLLEGE 

2386 

284, 300 ( 

3X) 

50. 8( 

1.1) 

49. 2( 

1.1) 

0.0 

276. 6( 

1.4) 

273. 1( 

1.1) 

UNKNOWN 

574 

SS,504( 

5X) 

51. 1( 

1.9) 

48. g( 

1.9) 

0.0 

247. 0( 

2.5) 

240. 6( 

2.3) 

*  CV  is  th«  cotffici«nt  of  variation  tor  the  sum  of  the  weights. 
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Table  16-102 


Weighted  Response  Percentages  and  U.S.  History  Proficiency  Means  with  Standard  Errors 
Grade  8,  by  Derived  Race/ethnicity,  for  Main  U.S.  History  Samples 


DERIVED  RACK/ETHNICITY    (NAEF  ID:  DRACE) 


N 

WEIGHTED  N 

(CV)* 

WHITE 

BLACK 

HISPANIC 

ASIAN 

AMER  IHD 

UNCt.A'JSIFIED 

MISSING 

—    TOTAL  — 

731,766( 

IZ) 

71. 3( 

0.6) 

14  .5( 

0. 

10. 6  ( 

0.3) 

2.2( 

0.2) 

1.3( 

0.2) 

(•  t{  0.0) 

0.0 

270. 4( 

0.8) 

246. 0( 

1.5) 

244. 3( 

1.9) 

273. 3( 

2.6) 

248. 6( 

3.8) 

260.^(11.4) 

GENDER 

MALE 

3158 

362,343( 

21) 

72. 1( 

0.8) 

13. 2( 

0.6) 

11. 5( 

0.5) 

2. 1( 

0.2) 

1.0( 

0.3) 

O.K  0.1) 

0.0 

272. 7( 

1.2) 

248. 8( 

2.1) 

245. 6( 

2.4 ) 

272. 8( 

5.0) 

250. 8( 

8.0) 

255.3(16.9) 

FEMALE 

3304 

369,423( 

21) 

70. 5( 

0.8) 

15. 8( 

0.6) 

9.  7( 

0.4) 

2.3( 

0.3) 

1.6( 

0.3) 

0.1(  0.0) 

0.0 

268. 1( 

0  .9) 

243. 8( 

1.7) 

242. 9( 

2.0) 

273. 8( 

3.4) 

247. 2( 

5.0) 

266.0(13.7) 

RACE/ETHNICITY 

WHITE 

4006 

521,546( 

IX) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0(  0.0) 

0.0 

270. 4( 

0.8) 

BLACK 

1005 

106,085( 

3X) 

0.0( 

0.0) 

100. 0< 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0(  0.0) 

0.0 

246. 0( 

1.5) 

HISPANIC 

1068 

77 , 623( 

3X) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0 . 0(  0.0) 

0.0 

244. 3( 

1.9) 

OTHER 

293 

26,513( 

7%) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

60. 6( 

4.7) 

36. 1< 

4.6) 

3.3(  1.0) 

0.0 

273. 3( 

2.6) 

248. 6< 

3.8) 

260.5(11.4) 

PARENTAL  EDUCATION 

LESS  THAN  H.S. 

587 

58,381( 

4X) 

53. 5( 

2.5) 

16. g( 

1.8) 

24. 7( 

2.3) 

1.7( 

0.6) 

2.g< 

0.8) 

0.4(  0.2) 

0.0 

249. 6( 

2.6) 

238. 7( 

2.9) 

238. 4( 

3.5) 

270.5(18.0) 

233.4(13.7) 

255.6(11.0) 

GRADUATED  H.S. 

1703 

196,308( 

3X) 

70. 6( 

1.2) 

16. 7( 

0.9) 

9.9( 

0.7) 

0.9( 

0.3) 

1.7( 

0.4) 

0.2(  0.1) 

0.0 

261. 4( 

1.5) 

240. 6( 

2.3) 

245. 0( 

2.5) 

258. 9( 

5.6) 

248. 7( 

6.9) 

275.3(12.8) 

SOME  EDUC  AFTER  H.S. 

1195 

135,631( 

3X) 

73. 1( 

1.4) 

15. 3( 

1.1) 

8.9( 

0.9) 

1.8( 

0.3) 

1.0( 

0.3) 

O.K  0.0) 

0.0 

274. 6( 

1.3) 

254. 0( 

1.9) 

249. 6( 

2.9) 

277. 3( 

7.5) 

260.1(10.7) 

220.5(35.2) 

GRADUATED  COLLEGE 

2386 

284,300( 

3X) 

79. 4( 

1.1) 

10. 8( 

0.8) 

6.1( 

0.6) 

2.9( 

0.3) 

0.8( 

0.2) 

O.K  0.0) 

0.0 

279. 0( 

0.9) 

255. 2( 

2.4) 

253. 8( 

2.9) 

283. 2( 

3.2) 

259. 5( 

7.8) 

281. 2(  5.5) 

UNXNOWIi 

574 

55,504( 

5X) 

47. 4( 

2. 6) 

20. 4( 

2.2) 

25. 8( 

1.9) 

4.8( 

1.1) 

1.6( 

0.6) 

0.0(  0.0) 

0.0 

254. 0( 

2.6) 

230. 3( 

2.3) 

233. 8( 

2.7) 

257. 4( 

5.7) 

236. 6( 

8.4) 

*  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 


493 


461 


49  d 


Table  16-103 


Weighted  Response  Percentages  and  U.S.  History  Proficiency  Means  with  Standard  Errors 
Grade  8,  by  Parental  Education,  for  Main  U.S.  History  Samples 


PARENTS*  EDUCAVION    (KAEP  ID:  PARED) 

N        WEIGHTED  H  (CV)< 
TOTAL  —  730,125(  IX) 


NOT  HS 

8.0(  0.3) 
244. g(  1.7) 


GRAD  HS 

26. g(  0.8) 
256. 1(  1.2) 


POST  HS 

18. 6(  0.6) 
269. 1(  1.1) 


GRfJ)  COL 

38. g(  0.9) 
274. 9(  0.9) 


UNKNOWN 


7.6( 
243. 8( 


0.4) 
1.8) 


MISSING 
0.2 


GENDER 
HALE 

FEMALE 


3145 
3300 


361.170(  2X) 
368,954(  22) 


6.2(  0.4) 

245. 0(  2.7) 

9.8(  0.6) 

244. 8(  2.1) 


27. 2(  1.0) 

257. 5(  1.9) 

26. 6(  0.9) 

254. 7(  1.2) 


18. 8(  0.8) 

272. 6(  1.9) 

18. 3(  0.7) 

265. 6(  1.5) 


40. 0(  1.0) 

276. 6(  1.4) 

37. 9(  1.2) 

273. 1(  1.1) 


7.9(  0.5) 

247. 0(  2.5) 

7.4(  0.4) 

240. 6(  2.3) 


0.3 
0.1 


RACE/ETHNICITY 
WHITE 

BLACK 

HISPANIC 

OTHER 

PARENTAL  EDUCATION 
LESS  TH?j;  tf.5j. 

GRADUATED  H.S. 

SOKE  EDUC  AFTER  H.S. 

GRADUATED  COLLEGE 

UNKNOWN 


4000 

520, 843 ( 

IX) 

6.0( 

0.4) 

26. 6( 

0.9) 

19. 0( 

0.7) 

43. 3( 

1.2) 

5.1( 

0.3) 

0.1 

249. 6( 

2.6) 

261. 4( 

1.5) 

274. 6( 

1.3) 

279. 0( 

0.9) 

254. 0( 

2.6) 

1090 

105.565( 

3X) 

9.4( 

1.1) 

31. 1( 

1.6) 

19. 6( 

1.4) 

29. 2( 

1.8) 

10. 7( 

1.2) 

0.5 

238. 7( 

2.9) 

240. 6( 

2.3) 

254. 0( 

1.9) 

255. 2( 

2.4) 

230. 3( 

2.3) 

1066 

77,489( 

3X) 

18. 6( 

1.8) 

25. 1( 

1.7) 

15. 5( 

1.7) 

22. 4( 

2.0) 

18. 5( 

1.6) 

0.2 

238. 4( 

3.5) 

245. 0( 

2.5) 

249. 6( 

2.9) 

253. 8( 

2.9) 

233. 8( 

2.7) 

288 

26, 228 ( 

7X) 

11. 0( 

1.9) 

20. 8( 

2.7) 

14. 6( 

2.4) 

40. 0( 

2.7) 

13. 6( 

2.5) 

1.1 

247.0(10.7) 

253. 8( 

5.2) 

270. 0( 

5.5) 

278. 2( 

2.8) 

252. 1( 

5.1) 

587 

58,381( 

4X) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0/ 

0.0( 

0.0) 

0.0( 

0.0) 

0.0 

244. 9( 

1.7) 

1703 

196,308( 

3X) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0( 

0.0) 

0.0( 

0.0) 

0.0 

256. 1( 

1.2) 

1195 

135,631( 

3X) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0,SJ) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0 

269. 1( 

1.1) 

2386 

284,300( 

3X) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

O.f ) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0 

274. 9( 

0.&) 

574 

55,504( 

5X) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0 

243. &: 

1.8) 

*  CV  is  th«  coefficient  of  vari&tior  for  the  sum  of  the  weights. 
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Table  16-104 


Weighted  Response  Percentages  and  U.S.  History  froficiency  Means  with  Standard  Errors 
Grade  12,  by  Gender,  for  Main  U.S.  History  Samples 


GEHDER  OF  SUBJECT    (NAEP  ID:  SEX) 


N 


WEIGHTED  N  (CV)* 


MALE 


FEMALE 


MISSIHG 


—    TOTAL  — 

5507 

677  105f  IXi 

47. 

7( 

1.3) 

52. 3( 

1.3) 

0.0 

298. 

5( 

1.3) 

291. 

8( 

1.1) 

0( 

MALE 

2593 

322  977^  3X5 

100. 

0( 

0.0) 

0. 

0.0) 

0.0 

298. 

5( 

2914 

354  128(  31) 

0. 

0( 

0.0) 

100. 

0( 

0.0) 

0.0 

7.91. 

8( 

1.1) 

PAPF /FTHNTPTTY 

riallt. 

3730 

500  0?A(  \X\ 

48 

0( 

1  41 

52 

Q( 

1.4) 

0.0 

305 

3( 

1.4) 

297 

2( 

1.3) 

975 

92,352(  3X) 

47 

5( 

1.8) 

52 

5( 

1.8) 

0.0 

276. 6( 

2.5) 

272 

4( 

1.9) 

HISPANIC 

616 

56,0A7(  5X) 

42 

8( 

2.8) 

57 

2( 

2.8) 

0.0 

274 

7( 

2.3) 

273. 2( 

2.3) 

OTHER 

186 

28,681(  8X) 

52 

0( 

3.2) 

48 

0( 

3.2) 

0.0 

291. 8( 

7.6) 

303 

4( 

5.7) 

PARENTAL  EDUCATION 

LESS  THAN  H.S. 

518 

54,278(  7X) 

41. 7( 

2.5) 

58 

3( 

2.5) 

0.0 

275 

8( 

2.8) 

273 

K 

3.0) 

GRADUATED  H.S. 

1379 

166,473(  3X) 

47. 6( 

2.0) 

52 

4( 

2.0) 

0.0 

287 

.8( 

1.6) 

283 

.0( 

1.6) 

SOME  EDUC  AFTER  H.S. 

1344 

162,888(  3X) 

45 

.6( 

2.0) 

54 

.4( 

2.0) 

0.0 

300 

.5( 

1.8) 

293 

.7( 

1.4) 

GRADUATED  COLLEGE 

2119 

27/.223(  4X) 

49 

.4( 

1.6) 

50 

.6( 

1.6) 

0.0 

310 

.0( 

1.9) 

302. 1( 

1-7) 

UNKHOWK 

123 

13,361(12X) 

62 

.7( 

5.1) 

37 

.3( 

5.1) 

0.0 

264 

.4( 

5.5) 

258. 0( 

5.7) 

*  CV  is  th«  coefficieat  of  variation  for  the  sum  of  the  wiiights. 
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Table  16-105 


Weighted  Response  Percentages  and  U,S,  History  Proficiency  Means  with  Standard  Errors 
Grade  12,  by  Derived  .^ace/ethnicity ,  for  Main  U.S.  History  Samples 


DERIVED  RACE/ETHr:iCITY    (NAEP  ID:  DRACE) 

M        WEIGHTED  S  <CV)*        WHITE  BLACK              HISPANIC          ASIAH              AKER  IND  UNCLASSIFIED  MISSING 

-    TOTAL  -                           5507          677.103(  IX)         73. 8(  0.6)  13. 6(  0.4)         8.3(  0.4)         3.3(  0.3)         0.9(  0.2)  O.K  0.1)  0.0 

301. 1(  1.2)  274. 4(  1.7)      273. 9(  1.8)      299. 4(  7.3)      293. 1(  6.0)  264.6(36.8) 

^^JSS                                    2593          322,977(  3X)         74. 4(  0.9)  13, 6(  0.7)         7.4(  0.6)         3.3(  0<4)         0.9C  0.2)  0.2(  0.1)  0.0 

303. 3(  1,4)  276. 6(  2.3)      274. 7(  2.3)      293, 3(  9,4)      291. 4(  6.0)  264.6(36.8) 

FEMALE                                 2914          334.128(  3X)         73. 4(  0.7)  13. 7(  0.7)         9.1(  0.4)         3.0(  0.3)         0.0(  0.3)  0.0(  0.0)  0  0 

297. 2(  1.3)  272. 4(  1.9)      273. 2(  2.3)      303. 9(  P  6)      294. 7(  9.8) 

RACE/ETHNICITY 
WHITE 

BLACK 

HISPANIC 

OTHER 


3730 

300,024 ( 

IX) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

O.G* 

0,0) 

0.0(  0.0) 

0.0 

301. 1( 

1.2) 

0.0(  0.0) 

0.0 

075 

92,332( 

3t) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

274. 4( 

1.7) 

0.0(  0.0) 

0.0 

616 

36.047( 

3X) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

273. 9( 

1.8) 

0.0 

186 

28»681( 

9X) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

77. 2( 

4.6) 

20. 8( 

4.3) 

2.1  1.3) 

299. 4 ( 

7.S) 

293. 1( 

6.0) 

264.6(36.8) 

^^^HAn"h^s!^^                    318            34,278(  7X)         30.8(  3.3)  16.n(  2.4)  26.9(  2.6)  4.1(  1.8)  1.3(  O.C)  0.0(  0.0)  0.0 

282. 3(  3.7)  261. g(  3.3)  263. 8(  3.8)  278.8(17.3)  271. 6(  8.6) 

1379          166.473(  3X)         74. 0(  1.2)  16. 4(  1.0)  6.3(  0.6)  1.9(  0.4)  1.1(  0.3)  0.1(  0.1)  0.0 

289. 8(  1.3)  270. 2(  1.9)  272. 3(  4.4)  286. 8(  7.7>  277. 6(  7.0)  273. 0(  7.7) 

SOME  EDUC  AFTER  H.S.            1344          162,888(  3X)         73. 4(  1.2)  13. 1(  1.1)  6.3(  0.6)  2.2(  0.6)  0.9(  0.3)  0.0(  0.0)  0.0 

tMu,  Luuu  ATiLn                                                              ^^^^^  ^^^^^  ^^^^^  293.1(11.0)  292.3(16.2) 

GRADUATED  COLLEGE                2119          277,223(  4X)         79. 4(  1.0)  10. 1(  0.6)  3.3(  0.3)  4.3(  0.7)  0.7(  0.3)  0-0(  0.0)  0.0 

unAuuAii^                                                                         ^^^^^  ^^^^^  g  2)  316. 2(  9.1)  286.9(****) 

123            13,361(12X)         33.4(6.0)  24.3(3.3)  33.9(3.3)  6.1(2.7)  0.0(0.0)  .0(0.0)  0.0 

283. 9(  8.0)  248. 7(  7.0)  232. 0(  3.0)  244.1(11.3) 


G'/AOUATED  H.S. 


UNKNOWN 


*  CV  la  th«  co«fflcl«nt  of  varlttlon  for  th«  lum  ct  th«  wtlshti. 
(****)  Standard  error  is  srtatar  than  99.9 
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Table  16-106 


Weighted  Response  Percentages  and  U.S.  History  Proficiency  Means  with  Standard  Errors 
Grade  12,  by  Parental  Education,  for  Main  U.S.  History  Samples 


PARENTS'  EDUCATIOH    (NAEP  ID:  PARED) 

N        WEIGHTED  N  (CV)*         KOT  HS  GRAD  HS  POST  HS  GRAD  COL  UNKKOWJI  MISSING 

—    TOTAL  —                            5483          674.223(  IX)           8.1(  0.6)  2*.7(  0.9)  2*.2(  0.8)  l.A)  2.0(  0.2)  0.4 

274. 2(  2.0)  285. 3(  1.1)  296. 8(  1.1)  306. 0(  1.6)  262. 0(  3.9) 


GENDER 


MALE 

2580 

321,479( 

3X) 

7 

0( 

0 

7) 

24 

7( 

1 

3) 

23 

K 

1 

1) 

42 

6( 

1 

8) 

2 

6( 

0 

4) 

0.5 

275 

8( 

2 

8) 

28' 

8( 

1 

6) 

300 

5( 

1 

8) 

310 

0( 

1. 

9) 

264 

4( 

5 

5) 

FEMALE 

2903 

352, 744 ( 

3X) 

9 

0( 

0 

6) 

24 

7( 

0 

9) 

25 

K 

i 

0) 

39 

8( 

1 

5) 

1 

4( 

0 

2) 

0.4 

273. 

K 

3 

0) 

283 

0( 

1 

6) 

293 

7( 

1 

4) 

302 

K 

1 

7) 

258 

0( 

5 

7) 

RACE/ETHNICITY 


WHITE 

3717 

498,532( 

IX) 

5. 

5( 

0 

6) 

24. 7( 

1.0) 

24 

6( 

0. 

9) 

44. 2( 

1 

6) 

0.9( 

0.2) 

0.3 

282 

5( 

3 

7) 

289. 8( 

1.3) 

301 

0( 

1. 

3) 

310. 5( 

1 

6) 

283. 9( 

8.0) 

BLACK 

973 

92,195( 

3X) 

9 

9( 

1 

4) 

29. 7( 

1.5) 

26 

6( 

1. 

7) 

30. 2( 

2 

1) 

3.6( 

1.0) 

0-2 

261 

9( 

5 

3) 

270. 2( 

1.9) 

283 

2( 

2. 

8) 

277. 9( 

2 

9) 

248. 7( 

7.0) 

HISPANIC 

608 

55, 167 ( 

4X) 

26 

5( 

2 

6) 

19. 5( 

1.8) 

19 

2( 

1. 

7) 

26. 7( 

2 

2) 

8.2< 

1-5) 

1.6 

265 

8( 

3 

8) 

272. 5( 

4.4) 

281 

4( 

3. 

1) 

285. 9< 

3 

1) 

252. 0( 

5-0) 

OTHER 

185 

28, 328 ( 

8X) 

10 

3( 

3 

6) 

18. 0( 

2.8) 

17 

9( 

3 

8) 

50. 9( 

6 

2) 

2.9( 

1-3) 

1.2 

277 

C(ll 

4) 

283. 0( 

5.2) 

292 

9(10 

3) 

312. 3( 

6 

9) 

244.1(11.5) 

ARENTAL  EDUCATION 

LESS  THAN  H.S. 

518 

54, 278 ( 

7X) 

100 

0( 

0 

0) 

0.0( 

0.0) 

0 

0( 

0 

0) 

0.0( 

0 

0) 

0.0( 

0.0) 

0.0 

274 

2( 

2 

.0) 

GRADUATED  H.S. 

1379 

166,473( 

3X) 

0 

0( 

0 

0) 

100. 0( 

0.0) 

0.0( 

0 

0) 

0.0( 

0 

0) 

0.0( 

0.0) 

0.0 

285. 3( 

1.1) 

SCX^  EDUC  AFTER  H.S. 

1344 

162, 888 ( 

3X) 

0 

0( 

0 

0) 

0.0( 

0.0) 

100 

0( 

0 

0) 

0.0( 

0 

0) 

0.0( 

0.0) 

0.0 

296 

8( 

1 

1) 

GRADUATED  COLLEGE 

2119 

277, 223 ( 

4X) 

0 

0( 

0.0) 

0.0( 

0.0) 

0 

0( 

0 

0) 

100. 0( 

0 

0) 

0.0( 

0.0) 

0.0 

306. 0( 

1 

6) 

UNKNOWN 

123 

13,361(12X) 

0 

0( 

0 

.0) 

0.0( 

0.0) 

0 

0( 

0 

0) 

0.0( 

0 

0) 

100. 0( 

0.0) 

262. 0(  3.9) 


*  CV  is  the  coefficient  of  variation  for  the  sum  of  the  weights. 
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Table  16-107 


Weighted  Response  Per::entages  and  Geography  Proficiency  Means  with  Standard  Errors 
Grade  12,  by  Gender,  for  Main  Geography  Samples 


GENDER  OF  SUBJECT    CNAEP  ID:  SEX) 


WEIGHTED  N  (CV)* 


MALE 


FEMALE 


MISSIHG 


—    TOTAL  — 

3030 

371,737(  IX) 

47. 8( 

1.3) 

52. 2( 

1.3) 

0.0 

in  1  o/ 

1    c  \ 
1.0) 

285. 7( 

1.2) 

GENDER 

MALE 

1425 

177  542^  AX^ 

inn 

u  •  u  ^ 

n  <\( 

n  t\\ 

0 . 0 

301 . 2( 

FEMALE 

1605 

184  195^  22^ 

u .  U  V 

U .  U  J 

0 . 0 

RACE/ETHNICITY 

WHITE 

280,4g5(  2Z) 

47. 4( 

1.5) 

52. 6( 

1.5) 

0.0 

309. 8( 

1.6) 

293. 3( 

1.2) 

BUCK 

535 

50,197(  kX) 

43. 6( 

3.0) 

56. 4( 

3.0) 

0.0 

262. 3( 

2.6) 

255. 4( 

2.5) 

HISPANIC 

300 

25,966(  7X) 

50. 4( 

3.1) 

49. 6( 

3.1) 

0.0 

279. 1( 

5.5) 

264. 4( 

3.7) 

OTHER 

102 

15,079(10Z) 

63. 1( 

5.3) 

36. 9( 

5.3) 

0.0 

301. ec 

4.2) 

288. 1( 

6.8) 

PARENTAL  EDUCATION 

LESS  THAN  H.S. 

268 

28,609(  IX) 

;3S.2( 

3.4) 

60. 8( 

3.4) 

0.0 

271. 8( 

3.4) 

263. 9( 

2.9) 

GRADUATED  H.S. 

750 

90.037(  5X) 

49. 3( 

2.2) 

50. 7( 

2.2) 

0.0 

290. 5( 

2.4) 

276. 7( 

2.3) 

SOME  EDUC  AFTER  H.S. 

725 

84,847(  42) 

46. 3( 

1.6) 

53. 7( 

1.6) 

0.0 

303. 3( 

2.6) 

286. 4( 

1.7) 

GRADUATED  COLLEGE 

1191 

157.243(  3X) 

48. 5( 

2.2) 

51. 5( 

2.2) 

0.0 

314. 0( 

2.3) 

297. 1( 

2.0) 

UNKNOWN 

82 

9,552(12X) 

58. 3( 

5.4) 

41. 7( 

5.4) 

0.0 

258. 4( 

7.1) 

250. 9( 

9.7) 

ERIC 


*  CV  is  tho  coefficient  of  variation  for  the  sum  of  the  neishts. 
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Table  16-108 


Weighted  Response  Percentages  and  Geography  Proficiency  Means  with  Standard  Errors 
Grade  12,  by  Derived  Race/ethnicity,  for  Main  Geography  Samples 


DERIVED  RACE/ETHNICITY    U'AEP  ID:  DRACE) 


WEIGHTED  N  (CV)  -* 


WHITE 


BUCK 


HISP/\NIC 


ASIA'' 


AMER  IND  UNCLASSIFIED 


HISSING 


—    TOTAL  — 

3030 

371,737( 

It) 

75. 

5( 

0.8) 

13. 5( 

0.5) 

7. 

OK 

0. 

5) 

2,7( 

0,3) 

l.K 

0,3) 

0.3( 

0,1) 

0 

.0 

301. 

1( 

1.1) 

258. 4 ( 

2.0) 

271, 

8( 

3. 

9) 

298. 7( 

4.3) 

292. 5( 

4,8) 

291,0(22.6) 

GENDER 

MALE 

1A25 

177,542( 

AX) 

7A. 

9( 

1,3) 

12. 3( 

1.0) 

7, 

4/ 

0. 

7) 

3.3( 

0,6) 

1.6( 

0.4) 

0.5( 

0,3) 

0 

,0 

309. 

8( 

1.6) 

262. 3 ( 

2.6) 

279, 

1( 

5. 

5) 

304 .2( 

4.4) 

297, 5( 

6.1) 

295.7(30.0) 

FEMALE 

1605 

194,195( 

21) 

75, 

9( 

1.0) 

14  .6( 

0.6) 

6. 

6( 

0 

7) 

2.2( 

0.4) 

0,6( 

0.2) 

O.K 

0,1) 

0 

.0 

293. 

3( 

1.2) 

255. 4 ( 

2.5) 

264, 

4( 

3. 

7) 

290. 9( 

8,1) 

280. 5( 

8.2) 

267. 8( 

RACE/ETHNICITY 

WHITE 

2093 

280,495( 

21) 

100. 

0( 

0.0) 

0.0( 

0,0) 

0, 

0( 

c 

0) 

0.0( 

0.0) 

0.0( 

0,0) 

0.0( 

0,0) 

0 

.0 

301. 

K 

1.1) 

BLACK 

535 

50,197( 

U) 

0. 

0( 

0.0) 

100. 0( 

0,0) 

0. 

0( 

0 

0) 

O.Ol 

0  Oy 

0.0( 

0.0) 

0.0( 

0.0) 

0 

.0 

258. 4( 

2.0) 

HISPANIC 

300 

25,966( 

71) 

0. 

0( 

0.0) 

0,0( 

0,0) 

100, 

0( 

0 

0) 

0,0( 

0,0) 

0,0( 

0,0) 

0,0( 

0.0) 

0 

,0 

271. 

8( 

3 

9) 

OTHER 

102 

15,079(10Z) 

0. 

0( 

0,0) 

0,0( 

0.0) 

0. 

0( 

0 

0) 

67. 5( 

6.4) 

26. 0( 

5.9) 

6,5( 

3,5) 

0 

.0 

298, 7( 

4.3) 

292, 5( 

4.8) 

291.0(22,6) 

PARENTAL  EOUCATION 

LESS  THAN  H.S. 

268 

28,609( 

71) 

51. 

r( 

3,3) 

20. 4( 

2,5) 

22, 

0( 

2 

4) 

3.1( 

1.4) 

2,9( 

1.  , 

0.0( 

0,0) 

0 

,0 

276. 

i( 

3.1) 

248, 0( 

5,6) 

258, 

8( 

6 

2) 

278. 1( 

8.4) 

289.2(11,9) 

GRADUATED  H.S. 

750 

90,037( 

51) 

74. 

3( 

1.8) 

17, 3( 

1.3) 

5. 

5( 

0 

9) 

1.7( 

0.4) 

0,9( 

0.5) 

0.2( 

0,2) 

0 

,0 

291. 

i( 

1,7) 

254. 5( 

4.8) 

270. 

7( 

3 

7) 

285. 5( 

Q.9) 

290,6(13.7) 

284,7(10,1) 

SOME  EDUC  AFTER  H.S. 

725 

84,847( 

«) 

75. 

8( 

1.5) 

14. 9( 

1.2) 

6. 

8( 

1 

1) 

1.9( 

0,8) 

0.6( 

0,4) 

0,0( 

0,0) 

0 

,0 

301. 

2( 

1.6) 

263. 1( 

3,4) 

279. 

5( 

7 

3) 

310. 1( 

7.1) 

314,4(17,7) 

GRADUATED  COLLEGE 

ll&l 

157,2A3( 

3X) 

82. 

2( 

1,2) 

8,4( 

0,7) 

4. 

5( 

0 

5) 

3.5( 

0,5) 

l.K 

0.4) 

0.3( 

0,2) 

0 

,0 

310. 

5( 

1,5) 

267, 0( 

4.1) 

281. 

9( 

5 

5) 

309, 0( 

5,4) 

288. 9( 

6,6) 

324,0(41.7) 

UNKNOWN 

82 

9>552(12Z) 

46. 

6( 

6,5) 

26. 2( 

.^.4) 

16. 

0( 

3 

7) 

7.5( 

2.4) 

0.0( 

0.0) 

3,7( 

3.7) 

0 

.0 

264. 

5(10.9) 

241. 2( 

8,0) 

255. 

K 

8 

1) 

248.6(20.5) 

251. 9( 

*  CV  is  the  coefficient  o£  variation  for  the  sum  of  the  Heights, 
standard  error  is  sweater  than  99.9. 
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Table  16-109 


Weighted  Response  Percentages  and  Geography  Proficiency  Means  with  Standard  Errors 
Grade  12,  by  Parental  Education,  for  Main  Geography  Samples 


PARENTS'  EDUCATION    (NAEP  ID:  PARED) 

N        WEIGHTED  K  (CV)* 
—    TOTAL  —  3016  370,289(  IX) 


nOT  HS 

7.7(  0.8) 
267. 0(  2. A) 


GRAD  HS 

24. 3(  1.1) 
263. 5(  1.6) 


POST  HS 

22. g(  0.0) 
294. 2(  1.6) 


GRAD  COL 

A2.5(  1.2) 
305. 3(  1.5) 


UNKNOWN 

2.6(  0.3) 
255. 2(  5.6) 


HISSING 
0.4 


GENDER 
MALE 

FEMALE 


1417 
1599 


176,688(  4Z> 
193.601(  2Z) 


6.4(  0.6) 

271. 8(  3.4; 

9.0(  0.7) 

263. 9(  2.9) 


25. 1(  1.4) 

290. 5(  2.4) 

23. 6(  1.4) 

276. 7(  2.3) 


22. 2(  1.2) 

30D.3(  2.6) 

23. 5(  1.1) 

286. 4(  1.7) 


43. 1(  1.6) 

314. 0(  2.3) 

41. 8(  1.8) 

297. 1(  2.0) 


3.2(  0.5) 

258, 4(  7.1) 

2.1(  0.3) 

250. 9(  9.7) 


0.5 
0.3 


RACE/ETHNICITY 
WHITS 

BLACK 

HISPANIC 

OTHER 


2087 
531 
296 
102 


279,707(  2X) 
49,882(  4Z) 
25,622(  7Z) 
15,079(101) 


5.3(  0.6) 

276. 1(  3.1) 

11. 7(  1.5) 

248. 0(  5.6) 

24. 5(  2.7) 

258. 8(  6.2) 

11. 2(  3.2) 

283. 5(  7.3) 


2b. 9(  1.4) 

291. 1(  1.7) 

31. 3(  2.1) 

254. 5(  4.8) 

19. 3(  2.5) 

270. 7(  3.7) 

16. 9(  4.0) 

287. 1(  6.0) 


23. 0(  1.0) 

301. 2(  1.6) 

25. 4(  2.2) 

263. 1(  3.4) 

22. 5(  2.7) 

279. 5(  7.3) 

13. 7(  4.0) 

311. 1(  5.8) 


46. 2(  1.5) 

310. 5(  1.5) 

26. 6(  2.0) 

267. 0(  4.1) 

27. 6(  2.7) 

281. 9(  5.5) 

51. 2(  4.8) 

305. 2(  4.9) 


1.6(  0.3) 
264.5(10.9) 

,'^.0(  1.2) 
241. 2(  8.0) 

6.0(  1.1) 
255. 1(  8.1) 

7.1(  2.7) 
249.7(15.5) 


0.3 
0.6 
1.3 
0.0 


PARENTAL  EDUCATION 


LESS  THAN  H.S. 

268 

28,609(  7Z) 

100. 0( 
267. 0( 

0.0) 
2.4) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0 

GRADUATED  H.S. 

750 

90,037(  5Z) 

0.0( 

0.0) 

100. 0( 
283. 5( 

0.0) 
1.8) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0 

SOetE  EDUC  AFTER  H.S. 

725 

84,847(  4Z) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 
294. 2( 

0.0) 
1.6) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0 

GRADUATED  COLLEGE 

1191 

157.243(  3Z) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

100. 0( 
305. 3( 

0.0) 
1.5) 

0.0( 

0.0) 

0.0 

UNKKOm 

82 

9,552(I2Z) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

0.0( 

0.0) 

100.0' 
255. 2C 

0.0) 
5.6) 

0.0 

*  CV  Is  tht  coefficient  oi  variation  for  th*  turn  of  tha  waights. 
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Appendix  A 


CONSULTANTS  FOR  THE  DEVELOPMENT 
OF  1988  NAEP  OBJECTIVES  AND  ITEMS 


READING 

Learning  Area  Gommittee 

Peter  Mosenthal,  Syracuse  University,  Syracuse,  NY 
Richard  Venezky,  University  of  Delaware,  Newark,  DE 


Development  Consultants  and  Reviewers 

Jo  Beth  Allen,  Kansas  State  University,  Manhattan,  KS 

Arthur  Applebee,  National  Council  of  Teachers  of  English,  Urbana,  IL 

Michael  Axline,  University  of  Oregon,  Eugene,  OR 

Fernie  Baca,  University  of  Colorado,  Denver,  CO 

Richard  Beach,  University  of  Minnesota,  Minneapolis,  MN 

Barbara  Bianchi,  Paideia  School,  Atlanta,  GA 

Susan  Blank,  Manpower  Demonstration  Research  Corporation,  Brooklyn,  NY 
Robin  Butterfield,  Northwest  Regional  Educational  Laboratory,  Portland,  OR 
Robert  Calfee,  Stanford  University,  Stanford,  CA 
Jeanne  Chall,  Harvard  Graduate  School  of  Education,  Cambridge,  MA 
Carita  Chapman,  Chicago  Public  Schools,  Chicago,  IL 

Ruth  Coleman,  North  3ide  High  School  Mothers  Alumni  Club,  Fort  Wayne.  IN 

Chris tor pher  Connell,  Associated  Press,  Washington,  DC 

Larry  Coon,  McDonald's  Restaurant  ,  Henderson,  TX 

Bernice  Cullinan,  New  York  University,  New  York,  NY 

Mary  E.  Curtis,  Harvard  Graduate  School  of  Education,  Cambridge,  MA 

Jacqueline  Danzberger,  Youthwork  Inc.,  Arlington,  VA 

Martha  Darling,  Washington  Roundtable  Education  Study,  Bellevue,  WA 

Philip  DiStefano,  University  of  Colorado,  Boulder,  CO 

Terry  Dozier,  South  Carolina  State  Department  of  Education,  Columbia,  SC 
Priscilla  Drum.  University  of  California  at  Santa  Barbara,  Santa  Barbara,  CA 
William  Eller,  State  University  of  New  York  at  Buffalo,  Amherst,  NY 
Leo  Estrada,  University  of  California,  Los  Angeles,  CA 
Claryce  Evans,  Boston  Public  Schools,  Boston,  MA 

Marjorie  Farmer,  School  District  of  Philadelphia,  Philadelphia,  PA 

Roger  Farr,  University  of  Indiana,  Bloomington,  IN 

Edmund  Farrell,  University  of  Texas,  Austin,  TX 

Edward  Fry,  Rutgers  University,  New  Brunswick,  NJ 

Carol  Gibson,  National  Urban  League,  New  York,  NY 

Kenneth  Goodman,  University  of  Arizona,  Tucson,  AZ 

Naomi  Gordon,  Public  Schools  of  Brookline,  Lexington,  MA 

Donald  Graves,  University    '  New  Hampshire,  Durham,  NH 

Jean  Greenlaw,  North  Texas  i^:ate  University,  Denton,  TX 

Doris  Hankins,  Germantown  High  School,  Germantown,  TN 

Jerome  Harste,  University  of  Indiana,  Bloomington,  IN 
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David  Hayes,  University  of  Georgia,  Athens,  GA 

Paul  Heffeman,  Star  Market,  Newtonville,  MA 

Harold  Herber,  Syracuse  University,  Syracuse,  NY 

Jane  Holt,  Champlain  Valley  Union  High  School,  Hinesburg,  VT 

Shu-in  Huang,  City  of  Thornton,  Thornton,  CO 

Evaline  Khayat  Kruse,  Audubon  Junion  High  School,  Los  Angeles,  CA 
Judith  Langer,  University  cf  California,  Berkeley,  CA 
Diane  Lapp,  Boston  University,  Boston,  MA 

Herbert  J,  Lapp,  Jr.,  GPU  Nuclear  Corporation,  Parsippany,  NJ 
Ron  Lessnau,  Hamburger  University,  Oakbrook,  IL 
Ray  Marshall,  University  of  Texas,  Austin,  TX 

X Ills  A.  Miller,  Reading  Development  Seminars,  Minneapolis,  MN 

Charlas  Moody,  University  of  Michigan,  National  Alliance  of  Black  School 

Educators,  Ann  Arbor,  MI 

Edwin  Newman,  NBC  News,  New  York,  NY 

Pedro  Pedraza,  Jr.,  Hunter  College,  New  York,  NY 

Anthony  Petrosky,  University  of  Pittsburgh,  Pittsburgh,  PA 

Carolyn  Pollan,  State  of  Arkansas,  Fort  Smith,  AR 

Walter  L.  Powers,  School  District  #271,  Coeur  d'Alene,  ID 

John  Readance,  Louisiana  State  University,  Baton  Rouge,  LA 

Beverly  Roller,  Jefferson  County  Public  Schools,  Littleton,  CO 

Glenn  E.  Rotz,  Highland  Elementary  School,  Clarkson,  WA 

Sarah  Saint-Onge,  Godine  Publishing  Co.,  Boston,  MA 

Adan  C.  Salgado,  Johnston  High  School,  Austin,  TX 

S.  Jay  Samuels,  University  of  Minnesota,  Minneapolis,  MN 

Robert  Schreiner,  University  of  Minnesota,  Minneapolis,  MN 

John  Stewig,  University  of  Wisconsin,  Milwaukee,  WI 

Dorothy  Strickland,  Columbia  University,  New  York,  NY 

Robert  Tierney,  University  of  Illinois,  Champagne-Urbana,  IL 

Jaap  Tuinman,  Sitoon  Fraser  University,  Burnaby,  B.C.,  Canada 

Janet  Tully,  Marriott  Corporationnn,  Washington,  DC 

Richard  Vacca,  Kent  State  University,  Kent,  OH 

Rod  Vahl,  Central  High  School,  Davenport,  lA 

Sheila  Valencia,  University  of  Colorado,  Boulder,  CO 

Thomas  Vallejos,  University  of  Colorado,  Boulder,  CO 

Maria  Watkins,  University  of  Pennsylvania,  Graduate  School  of  Education, 
Philadelphia ,  F^,nn£?yj!.vania 

Rick  Wetherell,  North  Bend  High  School.  West  Bend,  OR 
Susan  M.  Wolf,  Tlie  Hastings  Center,  Hastings -on-Hudson,  NY 
Kathy  Yen,  San  Francisco  Public  Schools,  San  Francisco,  CA 
Seymour  Yesner,  Brookline  High  School,  Brookline,  MA 


WRITING 

Learning  Area  Committee 

Arthur  Applebee,  Stanford  University,  Stanford,  CA 
Mary  Barr,  California  State  Department  of  Education,  San  Diego,  CA 
David  Bartholomae,  University  of  Pittsburgh,  Pittsburgh,  PA 
Miriam  Chaplin, 
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Paul  Diehl,  University  of  Iowa,  Iowa  City,  lA 
Ed  Folsom,  University  of  Iowa,  Iowa  City,  lA 

Helen  Kelley,  State  Department  of  Education,  Commonwealth  of  Virginia, 
Richmond,  VA 


Development  Consultants  and  Reviewers 

Robert  Alligood,  Plymouth  Public  Schools,  Plymouth,  NC 
Lee  Antell,  Champlin,  MN 

Elsa  Bartlett,  New  York  University  Medical  Center,  New  York,  NY 

Opaline  Brice,  California  PTA,  Inglewood,  CA 

Bill  Bums,  Boulder  High  School,  Boulder,  CO 

Mary  Busch,  Indiana  Central  College,  Indianapolis,  IN 

Robin  Butterfield,  N.W.  Indian  Reading  Program,  Portland,  OR 

Shirley  Cain,  Golden,  CO 

Rosa  Casarez,  Do;jney,  CA 

Patricia  Candal,  University  of  South  C/iColina,  Columbia,  SC 
Courtney  Cazden,  Harvard  University,  Cambridge,  MA 
Gilbert  Cho,  Cambridge,  MA 

Jane  Christensen,  National  Council  of  Teachers  of  English  Urbana,  IL 
Nancy  Ciarleglio,  New  Haven  PTA,  New  Haven,  CT 

Edmond  Cody,  North  Side  Independent  School  District,  San  Antonio,  TX 

Charles  Co^n-   ,  University  of  San  Diego-La  Jolla,  La  Jolla,  CA 

Florence  Cox,  Chicago  Region  PTA,  Chicago,  IL 

Alonzo  Crim,  Atlanta  City  Schools,  Atlanta,  GA 

John  Daly,    ^liversity  of  Texas  at  Austin,  Austin,  TX 

Leonard  Da-uCon,  Washoe  City  School  District,  Reno,  NV 

Vivian  Davis,  Tri-Ethnic  Committee,  Dallas,  TX 

Johit  Dawkins,  Yard  ley,  PA 

Margaret  Dinn,  Dade  County  Elementary  &  Secondary  Schools,    Miami,  FL 
Joanne  Eresh,  West  Liberty  Training  Center,  Pittsburgh,  PA 
Marjorie  Farmer,  Schoo?  District  of  Philadelphia,  Philadelphia,  PA 

rcia  Farr,  University  of  Illinois  at  Chicago,  Chicago,  IL 
Paul  Fawson,  Weber  County  School  District,  Ogden, 
Eleanor  Francke,  Lincoln  Public  Schools,  Lincoln,  NE 
June  Gabler,  Woodhaven  School  District,  Romulus,  MI 
Darrell  Garber,  New  Orleans  Public  Schols,  New  Orleans,  LA 
Pete  Garcia,  Espanola  School  District,  Espan^la,  WM 
Perry  Gilmore,  University  of  Alaska  ac  Fairb.^nks,  Fairbanks,  AK 
Gene  Goff  Jr.,  West  Virginia  Congress  of  Parents  and  Teachers,  Inc.. 

Huntington,  WV 
Do\-.ald  Graves,  University  of  New  Hampshire,  Durham,  NH 
Karen  Greenberg    The  Ciuy  University  of  New  York,  New  York,  NY 
Carol  Ann  Greenhalgh,  Texas  Education  Agency,  Austin,  TX 
Robert  Gundlach,  Northwestern  University,  Evanston,  TL 
Kris  Gutierrez,  University  of  Colorado,  Boulder,  CO 
Carol  Harner,  University  of  Denver,  Denver,  CO 
Barbara  Hasek,  Oregon  State  PTA,  Springfield,  OR 
Mary  Heim,  Grant  Elementary  School,  Casper,  WY 
Diane  Hernandez,  Lafayette  Elementary  School,  Lafayette,  CO 
Wajme  Hill,  Board  of  Education,  Cumberland,  MD 
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Susan  Hogsgard,  Pittsburgh  Board  of  Education,  Pittsburgh,  PA 

Jack  Holmquist,  Nebraska  PTA,  York,  NE 

Shu-In  Huang,  City  of  Thornton,  Thornton,  CO 

Ann  Humes,  Southwest  Regional  Laboratory,  Las  Aiamitos,  CA 

Enid  Humphrey,  West  Side  Community  Schools,  Omaha,  NE 

Richard  Johnson,  Center  for  New  Schools,  Highland  Park,  XL 

Donald  Jones,  Jefferson  County  Public  Schools,  Lakewood,  CO 

Kenneth  Kantor,  University  of  Georgia,  Athens,  GA 

Jalma  Killian,  Wheat  Ridge,  CO 

James  Rex  Kirk  Sr.,  Utah  PTA,  Tooele,  UT 

Carl  Klaus,  University  of  Iowa,  Iowa  City,  lA 

Rita  Klemm,  Wheat  Ridge  Senior  High  School,  Wheat  Ridge,  CO 

Shari  Lahr,  Buckeye  Central  School  District,  New  Washington,  OH 

Judith  Langer,  University  of  California  at  Berkeley,     Berkeley,  CA 

Glenda  Liston,  Texas  Congress  of  Parents  and  Teacheers,  Marshall,  TX 

Wendy  Li ttlef air.  Measurement,  Inc.,  Durham,  NC 

Richard  Lloyd-Jones,  University  of  Iowa,  Iowa  City,  lA 

Fay  Loo,  Seward  Park  High  School,  New  York,  NY 

Dorothy  Magett,  Seattle  Public  Schools,  Seattle,  WA 

Betty  Mangum,  State  Department  of  Public  Instruction,  Raleigh,  NC 

Maureen  McCarthy,  Totten  Taterraediate  School,  Staten  Island,  NY 

George  McCulley,  Michigan  Tech  University,  Houghton,  MI 

Carol  Mathews,  Boulder  High  School,  Boulder,  CO 

Frances  McCormick,  Manning  Junior  High  School,  Golden,  CO 

Sister  Jarlath  McManus,  Archdiocese  of  Denver,  Denver,  CO 

Mary  Meier,  Eugene  School  District,  Eugene,  OR 

John  Mellon,  University  of  Illinois,  Evanston,  IL 

Patti  Mendes,  University  of  Colorado,  Boulder,  CO 

Vana  Meredith,  State  Department  of  Education,  Columbia  SC 

Donna  Miller,  Orange  County  Public  Schools,  Orlando, 

Sandra  Murphy,  University  of  California  at  Berkeley,     _rkeley,  CA 

Jeff  Oliver,  Lincoln  Elementary  School,  Boulder,  CO 

Robert  Palmacier,  Birmingham  Hoard  of  Education,  Birmingham,  AL 

Jan  Patton,  Findlay  City  Schools,  Findlny,  OH 

Jesse  Perry,  San  Diego  Public  Schools,  San  Diego,  CA 

Anthony  Petrosky,  University  of  Pittsburgh,  Pittsburgh,  PA 

Lorraine  Plasse,  Springfield  Public  Schools,  Springfield,  MA 

Sanford  Powell,  Northwest  Mississippi  Teacher  Center,  Senatobia,  NY 

Edys  Quellmaiz,  University  of  California  at  Los  /  .g^^es,  Los  Angeles,  CA 

Edward  Reidy,  West  Hartford  Public  Schools,  West  Hartford,  CT 

Edward  Roeber,  Michigan  Department  of  Education,  Lansing,  MI 

Marilyn  Rosenblat,  College  Learning  Laboratory,  Buffalo,  NY 

Thomas  Roy,  Ithaca  City  Schools,  Ithaca,  NY 

Charles  Schuster,  University  of  Wisconsin,  Milwaukee,  WI 

Evelyn  Scott,  Englewood,  CO 

Sandra  Seale,  Cherry  Creek  High  School,  Aurora,  CO 

Sharif  Shakrani,  Michigan  Department  of  Education,  Lansing,  MI 

Mary  Ann  Shea,  University  of  Colorado,  Boulder,  CO 

Yvonne  Siu-Runyan,  Boulder  Valley  Schools,  Pasadena,  CA 

Robert  Smith,  Sandusky  City  Schols,  Sandusky,  OH 

Geneva  Smitherman,  Wayne  State  University,  Detroit,  MI 

Susan  Sowers,  Cambridge,  MA 

474 


ERIC 


Mary  Stitt,  Jefferson  County  Public  Schools,  Lakewood,  CO 
Barbara  Thompson,  Wisconsin  Department  of  Public  Ins' x.-uction , 
Donnd  Tovmsend,  Texas  Education  Agency,  Austin,  TX 
Lynn  Troyka,  CUNY,  Queensborough  Community  College,  Bayside, 
Tomas  Vallejos,  University  of  Minnesota,  Minneapolis,  MN 
Faith  Waters,  Bucks  County  School  District,  Doylestown,  PA 
Maria  Watkins,  Philadelphia,  PA 

Richard  Waaver,  Bentonville  Schools,  BentonviUe,  AR 
Ben  Willi.v.-r,  Chicago  Board  of  Education,  Chicago,  IL 
Darnell  Williams,  Bishop  College,  Dallas,  TX 
John  Wood,  Jr.chem  Elementary  School,  Broomfield,  CO 
Seymour  Yesner,  Brookline  Education  Center,  Brookline,  MA 


GEOGRAPHY 

Learning  Area  Committ-ae 

Russ  Allen,  Wisconsin  Department  of  Public  Instruction,  Madison, 

Norman  Bettis,  Illinois  State  University,  Normal,  IL 

Marianne  Kennev,  Cherry  Creek  Schools,  Denver,  CO 

Dana  Kurfman,  Prince  Georges  County  Schools,  Landover,  MD 

Christopher  Salter,  National  Geographic  Society,  Washington,  DC 


Development  Consultants  and  Reviewers 

Alan  Backler,  Indiana  University,  Bloomington,  IN 

Phillip  Bacon,  University  of  Houston,  Houston,  TX 

Richard  Boehm,  Texas  State  University,  San  Marcos.  TX 

Nancy  Brown,  Mississippi  Department  of  Education,  Jackson,  MS 

Gary  Elbow,  Texas  Tech  University,  Lubbock,  TX 

Charles  Gritzner,  South  Dakota  State  University,  Bronkings,  SD 

Michael  Hartoonian,  Wisconsin  Department  of  Public  Instruction.  Madison,  WI 

Richard  Heiboci-.,  La  Fosta:  Journal  of  American  Postal  History,  West  Linn,  OR 

Gail  Hobbs,  University  of  California,  Los  Angeles,  CA 

Briavel  Holcomb,  Rutgers  University,  New  Brunswick,  NJ 

Oscar  Horst,  Western  Michigan  University,  Kalamazoo,  MI 

Mary  Jean  Katz,  Portland,  OR 

Gail  Ludwig,  University  of  Missouri,  Columbia,  MO 

James  Harran,  New  Trier  Township  High  So'   ol.  Winnetka,  IL 

Gary  Miller,  F.W.  Cox  High  School,  Virginia  Beacn,  VA 

William  Miller,  Louisiana  Department  of  Education,  Baton  Rouge.  1« 

Janice  Honk,  University  of  Arizona.  Tucson,  AZ  „  ^  nr 

Salvatore  Natoli,  National  Council  for  the  Social  Studies,  Washington.  DC 

Fleta  Nockels,  Dogulas  County  Schools,  Castle  Rock,  CO 

Douglas  Phillips,  Anchorage  School  District,  Anchorage,  AK 

Sandra  Pritchard,  West  Chester  University,  We-t  Chester,  PA 

William  R.  Strong,  University  of  North  Alabai    .  Florence,  AL 

Doug  Wilms,  East  Carolina  University,  Greenville,  NC 

Barbara  Winston,  Northeastern  Illinois  University.  Chicago,  IL 

475 

ERIC  50.9 


U,S.  HISTORY 


Learning  Area  Committee 

Mary  Graham,  Metropolitan  Public  Schools,  Nashville,  TN 
Joel  Grossman,  University  of  Wisconsin,  Madison,  WI 
David  Hanmiack,  Case  Western  Reserve  University,  Cleveland,  OH 
Peter  Kneedler,  California  Department  of  Education,  Sacramento,  CA 
Sheila  Mann,  American  Political  Science  Associates,  Washington,  DC 
Sam  Natoli,  National  Council  for  the  Social  Studies,  Washington,  DC 
Harry  Scheiber,  Chair,  University  of  California,  Berkeley,  CA 
Betty  Waugh,  West  Mesa  High  School,  Albuquertue,  NM 


State  Advisory  Committee 

Russ  Allen,  Wisconsin  Department  of  Public  Instruction,  Madison,  WI 

Helen  Brown,  Bureau  of  Curriculum,  In-Service  &  State  Dept.,  Baton  Rouge,  LA 

John  Chapman,  Michigan  State  Department  of  Education  Lansing,  MI 

Roger  Hammer,  Wyoming  Department  of  Education,  Cheyenne,  WY 

Peter  Kneedler,  California  Department  of  Education,  Sacramento,  CA 

Douglas  Robertson,  State  Department  of  Public  Adminstration,  Raleigh,  NC 

Lois  Rubin,  Virginia  Department  of  Education,  Richmond,  VA 


Development  Consultants  and  Reviewers 

Ron  Adams,  Kingswcod  High  School,  Wolfeboro,  NH 
James  F.  Adomanis,  Annapolis,  MD 
Susan  Austin,  Bala  Cynwyd,  PA 

Marvin  Awbrey,  Fresno  Unified  School  District,  Fresno,  CA 
Rosemary  Bane,  Robertson  Academy,  Nashville,  TN 
J.  Sherman  Barker,  The  Hotchkiss  School,  Lakeville,  CT 
Jim  Bell,  Poway  High  School,  Foway,  CA 

Marjorie  Bingham,  St.  Louis  Park  Schools,  St.  Louis  Park,  MN 

Jeanine  Blumberg,  SUNY  at  S^T,ny  Brook,  Stony  Brook,  NY 

Mabel  McKinney  Bowning,  YEFs.,,  American  Bar  Assn.,  Chicago,  IL 

Judy  Bristol,  Houston  Independent  School  District,  Houston,  TX 

Nancy  Brown..  Mississippi  Board  of  Education,  Jackson,  MS 

George  Burson,  Aspen  High  School,  Aspen,  CO 

Robert  Campbell,  Wesleyan  Church,  Lambertville,  NJ 

Mary  Ann  Cardia,  The  Philadelphia  School,  Philadelphia,  PA 

Carol  Chaet,  Cambridge  Public  Schools,  Cambridge,  MA 

Phyllis    larke.  Boulder  Valley  Schools,  Boulder,  CO 

Harriet  Crane,  The  Baldwin  School,  Bryn  Mawr,  PA 

Gerald  Danzer,  University  of  Chicago,  Chicago,  IL 

Robert  M.  Dawson,  Albuquerque,  NH 

Charlotte  de  Costa,  The  Baldwin  School,  Bryn  Mawr,  PA 
Cooper  Delk,  McMurray  Middle  School,  Nashville,  TN 

476 


510 


Deborah  Drucker,  Springbrook  High  School,  Chevy  Chase,  MD 
Terrie  Epstein,  Cherry  Creek  High  School,  Englewood,  CO 
William  Femekes,  Fleming  ton,  NJ 
George  Flittie,  Portland,  OR 

Mary  Giunta  (ex  officio),  NARA/NHPRC,  Washington,  DC 

C.  Dale  Greewald,  Learning  Improvement  Services,  Nederland,  CO 

David  Harris,  Oakland  County  Public  Schools,  Pontiac,  MI 

JoAnn  Heidenrich,  Hillwood  High  School,  Nashville,  TN 

Sylvia  Hoffert,  St.  Louis  Country  Day  School,  St.  Louis,  MO 

Reverend  Giles  P.  Hayes,  O.S.B.,  The  Delbarton  School,  Morristown,  NJ 

Linda  Glickstein,  Philadelphia,  PA 

David  Glickstein,  Philadelphia,  PA 

Paul  Denris  Hoffm''      Canyon  del  Oro  High  School,  Tucson,  AZ 
Donna  Hudson,  01        iter  Elementary  School,  Antioch,  TN 
Alison  Johnson,  Gxencliff  High  School,  Nashville,  TN 
Savannah  Jones,  Birmingham  Public  Schools,  Birmingham,  AL 
Sylvia  Kamowsky,  Overton  High  School,  Nashville,  TN 
Joseph  Kovacs,  Edison  School  District,  Edison,  NJ 

David  Laudenschlager,  Rapid  City  Central  High  School,  Rapid  City,  SD 

James  Loguidice,  Bucks  County  Intermediate  Unit,  Doylestown,  PA 

Thomas  Lyons,  Phillips  Academy,  Andover,  MA 

Anne  McNair,  Kirkpatrick  Elementary  School,  Nrshville,  TN 

Mary  McFarland,  Parkway  School  District,  Chesterfield,  MO 

David  Meisrow,  Portland,  OR 

Fay  Metcalf ,  National  Council  of  the  Social  Studies,  Washington,  DC 

S.  Rex  Morrow,  Pennsylvania  State  University,  University  Park,  PA 

Carla  Nankervis,  Meigs  Magnet  School,  Nashville,  TN 

Louise  Osborne,  Fall -Hamilton  Elementary  School,  Nashville,  TN 

Arthur  Pease,  Lebanon  High  School,  Lebanon,  NH 

Frederick       Pfister,  Bloomfield  Hills,  MI 

Douglas  Phillips,  Anchorage  School  District,  Anchorage,  AK 

John  Phillips,  Sacramento,  CA 

Catherine  Pickle,  Memphis  City  Schools,  Memphis,  TN 
Gene  Pickel,  Oak  Ridge  School,  Oak  Ridge,  TN 

Helen  Richardson,  Fulton  County  Board  of  Education,  Atlanta,  GA 
Susan  Roberts,  Albuquerque,  NM 

Lucille  Robinson,  Ontario-Montclair ,  Ontario,  CA 
Al  Rocca,  Sequoia  Junior  High  School,  Redding,  CA 
John  Rossi,  El  Cerrito  Senior  High,  El  Cerrito,  CA 
Jennie  Scott,  Moore  Middle  School,  Nashville,  TN 

Denny  Shillings,  Homewood-Flossmore  Community  High  School,  Flossmore,  IL 

Larry  Strickland,  Public  Instruction  Office,  Tumwater,  WA 

Ruth  Sykes,  Haywood  Elementary  School,  Nashville,  TN 

Elaine  M.  Takenaka,  Department  of  Education,  Honolulu,  HI 

Jan  Talbot,  Fair  Oaks,  CA 

Steven  Teal,  Hercules,  CA 

Nan  Teeter,  Percy  Priest  Elementary  School,  Nashville,  TN 

Paul  Tice,  McMurray  Middle  School,  Nashville,  TN 

Mary  Jane  Turner,  Center  for  Civic  Education,  Calabasas,  CA 

Jane  Vandercook,  Hunters  Lane  Comprehensive  High  School,  Hendersonville ,  TN 

Deborah  Welch,  AHA,  Washington,  DC 

Mary  Lou  Williams,  Santa  Fe,  NM 

477 

ER?C  5il 


Thomasine  Wilson,  Berkeley,  CA 

Virginia  Wilson,  North  Carolina  School  of  Science  and  Mathematics.  Durham,  NC 
Celeste  Woodley,  Boulder  Valley  Schools,  Boulder,  CO 
Brain  R.  Wright,  Neel's  Bend  Middle  School,  Nashville,  TN 


CIVICS 

Learning  Area  Committee 

Russ  Allen,  Wisconsin  Department  of  Public  Instruction,  Madison,  WI 
Mary  Graham,  Metropolitan  Public  Schools,  Nashville,  TN 
Joel  Grossman,  University  of  Wisconsin,  Madison,  WI 
David  Hammack,  Case  Western  Reserve  University,  Cleveland,  OH 
Peter  Kneedler,  California  Department  of  Education,  Sacramento,  CA 
Sheila  Mann,  American  Political  Science  Associates,  Washington,  DC 
Sam  Natoli,  National  Council  for  the  Social  Studies,  Washington,  DC 
Harry  Scheiber,  Chair,  University  of  California,  Berkeley,  CA 
Betty  Waugh,  West  Mesa  High  School,  Albuquerque,  NM 


State  Advisory  Committee 

Russ  Allen,  Wisconsin  Department  of  Public  Instruction,  Madison,  WI 

Helen  Brovm,  Bureau  of  Cuirlculum,  In-Service  &  Statt  ?  iC,  Baton  Rouge,  LA 

John  Chapman,  Michigan  Scate  Department  of  Education,  Lansing,  MI 

Roger  Hanuner,  Wyoming  Department  of  Education,  Cheyenne,  WY 

Peter  Kneedler,  California  Department  of  Education,  jacramento,  CA 

Douglas  Robertson,  State  Department  of  Public  Admirstration,  Raleigh,  NC 

Lois  Rubin,  Virginia  Department  of  Education,  Richmond,  VA 


Development  Consultants  and  Reviewers 

Ron  Adams,  Kingswood  High  School,  Wolfeboro,  NH 
James  F.  Adomanis,  Annapolis,  MD 
Susan  Austin,  Bala  Cynwyd,  PA 

Marvin  Awbrey,  Fresno  Unified  School  District,  Fresno,  CA 
Rosemary  Banc»  Robertson  Academy,  Nashville,  TN 
J.  Sherman  Barker,  The  Hotchkiss  School  Lakeville,  CT 
Jim  Bell,  Poway  High  School,  Poway,  CA 

Marjorie  Binghau,  St.  Louis  Park  Schools,  St.  Louis  Pa"^k,  MN 

Jeanine  Blumberg,  SbNY  at  Stony  Brook,  Stony  Brook »  NY 

Mabel  McKinney  Bowniug,  YEFC,  American  Bar  Assn.,  Chicago,  IL 

Judy  Bristol,  Houston  Independent  School  District,  Houston,  TX 

Nancy  Brov^,  Mississippi  Board  of  Education,  Jack.son,  MS 

George  Burson,   \spen  High  School,  Aspen,  CO 

Robert  Campbell,  Wecleyan  Church,  Lambertville,  NJ 

Mary  Ann  Cardia,  The  Philadelphia  School,  Philadelphia,  PA 

Carol  Chaet>  Cambridge  Public  Schools,  Cambridge,  MA 

Phyllis  Clarke,  Boulder  Valley  Schools,  Boulder,  CO 


478 


Harriet  Crane ,,/The    aldwin  School,  Bryn  Mawr,  PA 
Gerald  Danzer,  University  of  Chicago,  Chicago,  IL 
Robert       Dawson,  Albuquerque,  NM 

Charlotte  de  Costa,  The  Baldwin  School,  Bryn  Mawr,  PA 
Cooper  nelk,  McMurray  Middle  School,  Nashville,  TN 
Deborah  Drucker,  Springbrook  High  School,  Chevy  Chase,  MD 
Terrie  Epstein,  Cherry  Creek  High  School,  Englewood,  CO 
William  Femekes,  Flemington,  NJ 
George  Fllttie,  Portland,  OR 

Mary  Giunta  (ex  officio),  NARA/NHPRC,  Washington,  DC 

C*  Dale  Greenwald,  Learning  Improvement  Services,  Nederland,  CO 

David  Harris,  Oakland  County  Public  Schools,  Pontiac,  MI 

JoAnn  Heidenreich,  Hillwood  High  School,  Nashville,  TN 

Sylvia  Hoffert,  St.  Louis  Country  Day  School,  St.  Louis,  MO 

Reverend  Gilas  P.  Hayes,  O.S.B.,  The  Delbarton  School,  Morristown,  NJ 

Linda  Glickstein,  Philadelphia,  PA 

David  Glickstein,  Philadelphia,  PA 

Paul  Dennis  Hoffman,  Canyon  del  Oro  High  School,  Tucson,  AZ 
Donna  Hudson,  Old  Center  Elementary  School,  Antic ^h,  TN 
Alison  Johnson,  Glencliff  High  School,  Nashville,  TN 
Savannah  Jones,  Birmingham  Public  Schools,  Birmingham,  AL 
Sylvia  Karnowsky,  Overton  High  School,  Nashville,  TN 
Joseph  Kovacs,  Edison  School  District  Edison,  NJ 

David  Uudenschlager,  Rapid  City  Central  High  School,  Rapid  City,  SD 

James  Loguidice,  Bucks  County  Intermediate  Unit,  Doylestown,  PA 

Thomas  Lyons,  Phillips  Academy,  Andover,  MA 

Ann  McNair,  Kirkpatrick  Elementary  School,  Nashville,  TN 

Mary  McFarland,  Parkway  School  District,  Chesterfield,  MO 

David  Meisrow,  Portland,  OR 

Fay  Metcalf,  National  Council  of  the  Social  Studies,  Washington,  DC 

S*  Rex  Morrow,  Pennsylvania  State  University,  University  Park,  PA 

Carla  Nankervis,  Meigs  Magnet  School,  Nashville,  TN 

Louis  Osborne,  Fall-Hamilton  Elementary  School  Nashville,  TN 

Arthur  Pease,  Lebanon  High  School,  Lebanon,  NH 

Frederick  W.  Pfister,  Bloomfield  Hills,  MI 

Douglas  Phillips,  Anchorage  School  District,  Anchorage,  AK 

John  Phillips,  California  Department  of  Education,  Sacramento,  CA 

Catherine  Pickle,  Memphis  City  Schools,  Memphis,  TN 

Gene  Picke.1,  Oak  Ridge  School,  Oak  Ridge,  TN 

Helen  Richardson,  Fulton  County  Board  of  Education,  Atlanta,  OA 
Susan  Roberts,  Albuquerque,  KM 

Lucille  Robinson,  Ontario-Montclair ,  Ontario,  CA 
Al  Rocca,  Sequioa  Junior  High  School,  Redding  CA 
John  Rossi,  El  Cerrito  Senior  High,  El  Cerrito,  CA 
Jennie  Scott,  Moore  Middle  School,  Nashville,  TN 

Denny  Shillings,  Homewood-Flossmore  Community  High  School,  Flossmore,  IL 

Larry  Strickland,  Public  Instruction  Office,  Tumwiter,  WA 

Ruth  Sykes,  Haywood  Elementary  School,  Nashvi]le,  "^N 

Elaine  M.  Takenaka,  Department  of  Education,  Honolui'i.  HI 

Jan  Talbot,  Fair  Oaks,  CA 

Steven  Teal,  Hercules,  CA 

Nan  Teeter,  Percy  Priest  Elementary  School,  Nashville,  TN 

479 


Paul  Tice,  McMurray  Middle  School,  Nashville,  TN 

Mary  Jane  Turner,  Center  for  Civic  Education,  Calabasas,  CA 

Jane  Vandercook,  Hunters  Lane  Comprehensive  High  School,  Hendersonville ,  TN 

Deborah  Welch,  AHA,  Washington.  DC 

Joy  Weldon,  Bethlehem,  PA 

Mary  Lou  Williams,  Santa  Fe,  t<M 

Thomasine  Wilson,  Berkeley,  CA 

Virginia  Wilson,  NC  School  of  Science  and  M*:*-hematics,  Durham,  NC 
Celeste  Woedley,  Boul     '  Valley  Schools,  Be      ^r,  CO 
Brian  R.  Wright,  Neely's  Bend  Middle  School,  Nashville,  TN 


BACKGROUND  QUESTIONS  CONSULTANTS 

Valeria  Ford,  Office  of  Research  and  Student  Assessment,  DC  Public  Schools 
Susan  Fuhrman,  Eagleton  Institute  for  Politics,  Rutgers  University 
Barbara  Heyns,  Center  for  Applied  Social  Science  Research,  .    y  York  University 
Nancy  Karweit,  Center  for  the  Social  Organization  of  Schools,  Hopkins 

University 
Lorraine  McDonnell,  Rand  Corporation 

Vilma  Ortiz,  Manpower  Demonstration  Research  Corporation 
Michael  Smith,  School  of  Education,  Stanford  University 
Herb  Walberg,  University  of  Illinois  at  Chicago  Circle 


APPENDIX  B 

Distribution  of  Weight  Components  for  1988  NAEP  Samples 


ERIC 


481 


515 


Appendix  B 

DISTRIBUTION  OF  WEIGHT  COK^ONENTS  FOR  1988  NAEP  SAMPLES 


The  following  11  tables,  which  are  cited  throughout  Chapter  8. 
"Weighting  Procedures  and  Estimation  of  Sampling  Variance,"  show  the 
distribution  of  weight  components  for  the  1988  NAEP  samples,  including  base 
weights,  the  various  nonresponse  adjustment  factors,  trimraing  factors,  and 
poststratification  factors,  for  the  student  weights,  the  excluded  student 
weights  and  the  teacher- student  weights.    A  description  of  each  table  is  given 
in  Chapter  8,  sections  8.1.1  through  8.1.6. 
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Table  B-1 


Dlstrioution  of  Student  Base  Weights  -  Assessed  Students 


No.  of 

Standard 

25th 

71"th 

Sample 

Cases 

Mean 

Deviation 

Min, 

Percentile 

Median 

Percentile 

Max, 

Grade  4/Age  9  Main 

20312 

157. 

1 

63.5 

56. 

8 

7 

1 

215.7 

860 

5 

Grade  8/Age  13  Main 

36699* 

88. 

6 

45.4 

21. 

2 

61 

0 

11 

6 

120.1 

526 

0 

Grade  12/Ag8  17  Main 

32710 

72. 

9 

35.9 

20. 

7 

g 

✓ 

1 

100.4 

402 

7 

Age  9  Bridge  -  Books  51-56 

5188 

728. 

5 

428.7 

213. 

1 

396. 

2 

617 

6 

980.3 

4787 

3 

Age  9  Bridge  -  Book  91 

1274 

2159. 

2 

1259.0 

364. 

0 

1121. 

5 

1833 

9 

2971.5 

8187 

1 

Age  9  Bridge  -  Bock  92 

1240 

2145. 

0 

1272.7 

364. 

0 

1145. 

1  • 

1852 

9 

2940.9 

8187 

1 

Age  9  Bridge  -  Book  93 

1197 

2277 . 

7 

1593  .7 

639. 

3 

1149. 

3 

1873 

6 

2958.9 

13169 

7 

Age  13  Bridge  -  Books  51-56 

5500 

646. 

7 

245.7 

180. 

1 

480. 

4 

647 

1 

740.3 

3867 

.7 

Age  13  Bridge  -  Book  90 

1938 

1288. 

5 

444.0 

360. 

2 

984. 

5 

1281 

2 

1558.0 

3970 

.0 

Age  13  Bridge  -  Book  91 

1405 

1924. 

4 

630.0 

853. 

5 

1450. 

3 

1921 

8 

2220.8 

3593 

.9 

Age  13  Bridge  -  Book  92 

1281 

2057. 

7 

1031.0 

600. 

3 

1483. 

5 

1906 

1 

2496.2 

11603 

0 

Age  13  Bridge  -  Books  93 

1256 

1904. 

7 

810.9 

540. 

3 

1488. 

8 

1921 

8 

2249.5 

12149 

2 

Age  17  Bridge  -  Bocks  51-56 

4622 

492. 

0 

248.7 

166. 

1 

306. 

5 

405 

0 

609.7 

1552 

.4 

Age  17  Bridge  -  Books  61  -  67 

7052 

335 

9 

169.5 

95. 

1 

203. 

8 

281 

2 

447.6 

1075 

.3 

Age  17  Bridge  -  Book  90 

1786 

1017 

4 

484.3 

332 

3 

633. 

7 

854 

.3 

1300.9 

2328 

.5 

*  Number  in^iudeb  5  098  13-year-old  students  who  were  assessed  as  part  of  the  International  Assessment  of 
Mathematics  and  Science  (see  A  World  of  Differences.     An  International  Assessment  of  Mathematics  and  Science.  Technical 
Report    [King,  Bertrand,  &  Dupuls,  1989)). 
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Table  B-2 


Distribution  of  School  Nontesponse  Adjustments  -  Assessed  Studer'us 


No.  of 

Standard 

Sample 

Cases 

Mean 

Deviation 

Grade  4/Agc  9  Main 

20312 

1. 

108 

0.289 

Grade  8/Age  13  Main 

36699* 

1. 

150 

0.318 

Grade  12/Age  17  Main 

32710 

1  . 

O  0  0 

2.11 

0.450 

Age  9  Bridge  -  Books  51-56 

5188 

1. 

055 

0.172 

Age  9  Bridge  -  Book  91 

1274 

1. 

047 

0.152 

Age  9  Bridge  -  Book  92 

1240 

1. 

055 

0.167 

Age  9  Bridge  -  Book  93 

1197 

1 

057 

0.174 

Age  13  Bridge  -  Books  51- 

56 

55O0 

1 

002 

0.097 

Age  13  Bridge  -  Book  90 

1938 

1 

008 

0.117 

Age  13  Bridge  -  Book  91 

1405 

1 

006 

0.106 

Age  13  Bridge  -  Book  92 

1281 

1 

000 

0.065 

Age  13  Bridge  -  Books  93 

1256 

1 

.006 

0 . 107 

Age  17  Bridge  -  Books  51- 

56 

4622 

JL 

.154 

0.331 

Age  17  Bridge  -  Books  61 

-  67 

7052 

1 

.148 

0.325 

Age  17  Bridge  -  Book  90 

1786 

1 

.155 

0.331 

25th 

75  th 

{dill, 

Percentile 

Median 

Percentile 

Max, 

0.796 

1 

000 

1. 

000 

1. 

000 

3.174 

0.836 

1. 

000 

1. 

000 

1 

107 

2.847 

0.872 

1 

000 

1 

000 

1 

260 

3.045 

1.000 

1 

000 

1. 

000 

1 

000 

2.000 

1.000 

1 

000 

1 

000 

1 

000 

2.007 

1.000 

1 

000 

1 

000 

1 

000 

2.007 

1.000 

1 

000 

1 

000 

1 

000 

2.007 

0.760 

1 

.000 

I 

000 

1 

.000 

1.890 

0.775 

1 

000 

1 

000 

1 

.000 

1.913 

0.775 

1 

.000 

1 

000 

] 

.000 

1.913 

0.775 

1 

.000 

1 

000 

1 

.000 

1.913 

0.775 

1 

.000 

1 

000 

1 

.000 

1.913 

0.990 

1 

.000 

1 

.000 

1 

.005 

2.000 

0.990 

1 

.000 

1 

.000 

1 

.005 

2.000 

0.990 

1 

.000 

1 

.000 

1 

.930 

2.012 

*  Number  includes  5,098  13-year-old  students  who  were  assessed  as  pari  of  the  International  Assessment  of 
Mathematics  and  Science  (see  A  World  of  Differences.    An  International  Assessment  of  Mathematics  and  Science.  Technical 
Report    (King,  Bertrand,  &  Dupuis,  1989]). 
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Table  B-3 

Distribution  of  Session  Nonrespcr.se  Adjustmei..i.s  -  Assessed  Students 


No .  of 

S  t'AnHATH 

\j  wdiiVAdx  va 

Od  S>«79 

no\yi  a  t"  i  nT> 
lyc  vxdcxLii^ 

Min 

jr  tsx  v*e  iik"X  x^ 

Po      OT^  f 

AC  X  ueiH| 

Mav 

nT"^Ho  A /Acta    Q  Mxi  n 

N/A 

GrAd**               1*^  Main 

wi.  civic    U^XlgC     Xi.^  ildJLii 

N/A 

Gradp  1?/Ajta  17  Ma  in 

vi.  due    x.Am/            X  /    iid  JLii 

N/A 

AcTo    Q  Rt*iHtTft    -   RnnlrQ   SI  -  Sfi 

X  •  7 

X  •  \J\J\J 

1  onn 

1  nnn 

1  nnn 

9  nnn 

X.£.  1  '-r 

X  •  \J\J\J 

1  nnn 

1  nnn 

1  nnn 

1  QQ7 

X  .  7  7  / 

Acta    Q  RT*irlcA    -  Rnnir 

X  •  U.J  7 

1  nnn 

1  nnn 

1  nnn 

1  nnn 

1  QQ7 

Ace  9  Bridee  -  Book  93 

1197 

1.062 

0.210 

1.000 

1.000 

1.000 

1.000 

2.453 

Age  13  Bridge  -  Books  51-56 

51  0 

1.006 

0.050 

1.000 

1.000 

1.000 

1.000 

1.434 

Age  13  Bridge  -  Book  90 

1938 

l.Olt 

0.161 

1.000 

1.000 

1.000 

1.000 

2.961 

Age  13  Bridge  -  Book  91 

1405 

0.982 

0.099 

0.411 

1.000 

1.000 

1.000 

1.000 

Age  13  Bridge  -  Book  92 

1281 

1.000 

0.000 

1.000 

1.000 

1.000 

1.000 

1.000 

Age  13  Bridge  -  Books  93 

1256 

0.990 

0.151 

0.464 

1.000 

1.000 

1.000 

Age  17  Bridge  -  Books  51-56 

4622 

1.093 

0.266 

1.000 

1.000 

1.000 

1.000 

2.504 

Age  17  Bridge  -  Books  61  -  67 

7052 

1.068 

0.209 

1.000 

1.000 

1.000 

1.000 

2.000 

Age  17  Bridge  -  Book  90 

1786 

1.098 

0.260 

1.000 

1.000 

1.000 

1.000 

2.131 
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Table  5-4 


Distribution  of  Age-Only-Eligible  Students  Nonrespon^e  Adjustments  •  Ai>sessed  Students 


No.  of 

Standard 

25th 

75  th 

Sample 

Cases 

Meai 

Deviation 

Min. 

Percentile 

Median 

Percentile 

rickX  • 

Grade  4/A2e  9  Main 

23012 

1.010 

0 

087 

1 

.000 

1 

.000 

1 

.000 

1 

.000 

Grade  8/Aee  13  Main 

36699* 

1.007 

0 

097 

1 

.000 

1 

.000 

1 

.000 

1 

000 

'K  nfiP 

Grade  12 /Ace  17  Main 

32710 

1.011 

0 

160 

1 

.000 

1 

000 

1 

.000 

1 

000 

H  •  0  jO 

Aee  9  Bridge  -  Books  51-56 

5188 

0 

110 

1 

.000 

1 

.000 

1 

.000 

1 

.000 

Z  .  Z  /  J 

Age  9  Bridge  -  Book  91 

1274 

1.008 

0 

087 

1 

.000 

1 

000 

1 

000 

1 

000 

Age  9  Bridge  -  Book  92 

1240 

1.010 

0 

081 

1 

000 

1 

.000 

1 

000 

1 

000 

1.780 

Age  9  Bridge  -  Book  93 

1197 

1.022 

0 

120 

1 

.000 

1 

.000 

1 

000 

1 

000 

2.273 

Age  13  Bridge  -  Books  51-56 

5500 

1.000 

0 

000 

1 

.000 

1 

.000 

1 

000 

1 

000 

1.000 

Age  13  Bridge  -  Book  90 

1938 

1.000 

0 

000 

1 

.000 

1 

.000 

1 

000 

1 

000 

1.000 

Age  13  Bridge  -  Book  91 

1405 

1.000 

0 

000 

1 

.000 

1 

.000 

1 

000 

1 

000 

1.000 

Age  13  Bridge  -  Book  92 

1281 

1.000 

0 

000 

1 

.000 

1 

000 

1 

000 

1 

000 

1.000 

Age  13  Bridge  -  Books  93 

1256 

1.000 

0 

000 

1 

.000 

1 

000 

1 

GOO 

1 

000 

1.000 

Age  17  Bridge  -  Books  51-56 

4622 

1.003 

0 

065 

1 

.000 

1 

000 

1 

000 

1 

000 

2.270 

Age  17  Bridge  -  Books  61  -  67 

7052 

1.005 

0 

081 

1 

000 

1 

000 

1 

000 

1 

000 

2.270 

Age  17  Bridge  -  Book  90 

1786 

1.006 

0 

080 

1 

000 

1 

000 

1 

.000 

1 

000 

2.270 

*  Number  includes  5,098  13-year-old  students  who  were  assessed  as  part  of  the  International  Assessment  of 
M^.thematic3  and  Science  (see  A  World  of  Differences.    An  International  Assessment  of  Mathematics  and  Science.  Technical 
Report    [King.  Sertrand.  &  Dupuis,  1989]). 


ERJC  488 


94 


Table  B-5 

Distribution  of  Student  Nonresponse  Adjustments  -  Assessed  Students 


No.  of  Standard  25th  75th 


o  amp  X  e 

Cases 

Mean 

Deviation 

Min, 

Percentile 

Median 

Percentile 

Max. 

uraae  ^f/Age  y  nam 

0  Qni  0 

1 

080 

0 

062 

1 

.000 

1 

044 

1 

063 

1 

.101 

2 . 121 

n-yfiAts     Q  /Acta     1  )Ufo-f«^ 

braae  o/Age  ij  nam 

jobyy* 

1 

133 

0 

095 

1 

.000 

1 

073 

1 

112 

1 

159 

1.857 

biaae  iz/Age  i/  nam 

Q  O  "7  1  n 

Jz  /iU 

]. 

265 

0 

198 

1 

.000 

1 

135 

1 

216 

1 

320 

2.672 

Age  y  Driage  -  dooks  jI-jo 

Rl  QQ 

1 

077 

0 

053 

1 

.000 

1 

037 

1 

067 

1 

108 

1.  265 

Age  y  Drioge  -  dook  yi 

1  O  ■?  A 
lZ/4 

1 

090 

0 

069 

1 

.000 

1 

039 

1 

077 

1 

138 

1.258 

Age  y  j^riage  -  dook  yz 

1  0  A  n 

1 

088 

0 

091 

1 

.000 

1 

021 

1 

''  '5 

1 

112 

1.  373 

Aero    Q    R'T-fHcro    .    Rnnlr  Q*^ 
/Igc    7    DX.XUgc    "    DOOK.  yj 

1  1  07 

1  ly  / 

1 

081 

0 

092 

1 

.000 

1 

019 

1 

073 

1 

102 

1 . 667 

Age  13  Br?  Ige  -  Books  51-56 

5500 

1 

113 

0 

IC 

1 

000 

1 

052 

1 

091 

1 

140 

2.333 

Age  13  Bridge  -  Book  90 

1938 

1 

121 

0 

147 

1 

000 

1 

048 

1 

102 

1 

152 

3.667 

Age  13  Bridge  -  Book  91 

1405 

1 

108 

0 

117 

1 

000 

1 

046 

1 

068 

1 

138 

1.904 

Age  13  Bridge  -  Book  92 

1281 

1 

095 

0 

107 

1 

000 

1 

030 

1 

065 

1 

124 

2.000 

Age  13  Bridge  -  Books  93 

1256 

1 

121 

0 

137 

1 

000 

1 

053 

1 

097 

1 

161 

2.200 

Age  17  Bridge  -  Books  51-56 

4622 

1 

277 

0 

241 

1 

.055 

1 

167 

1 

223 

1 

296 

2.749 

Age  17  Bridge  -  Books  61  -  67 

7052 

1 

268 

0 

262 

1 

045 

1 

149 

1 

205 

1 

267 

3 . 14.'i 

Age  17  Bridge  -  Book  90 

1786 

1 

271 

0 

210 

1 

026 

1 

166 

1 

232 

1 

311 

2.078 

*  Number  includes  5,098  13-year-old  students  who  were  assessed  as  part  of  the  International  Assessment  of 
Mathematics  and  *^cience  (see  A  World  of  Diffezt^nces.     An  Iniemationai  Assessment  of  Mathematics  and  Science.  Technical 
Report     (King,  Bertrand,  &  Dupuis,  1989]). 
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Table  B-6 


Distribution  of  Trimming  Factors  -  Assessed  Students 


No.  of 

Standard 

25th 

75th 

Sample 

Cases 

Mean 

Deviation 

Min, 

Percentile 

Median 

Percentile 

Max. 

Grade  4/Age  9  Main 

23012 

0 

995 

0 

026 

0 

7:-9 

1 

000 

1.000 

1 

.000 

1.000 

Grade  8/Age  13  Main 

36699* 

0 

998 

0 

.022 

0 

72S 

1 

000 

1.000 

1 

000 

1.000 

Grade  12/Age  17  Main 

32710 

0 

998 

0 

021 

0 

760 

1 

000 

1.000 

1 

.000 

1.000 

Age  y  oridge  -  Books  jI-jd 

c  1  0  0 

0 

995 

0 

046 

0 

560 

1 

000 

1  f\f\f\ 

1 . 000 

1 

.000 

1.000 

Age  9  Bridge  -  Book  91 

1274 

0 

993 

0 

032 

0 

809 

1 

000 

1.000 

1 

000 

1.000 

Age  9  Bridge  -  Book  92 

1240 

0 

996 

0 

026 

0 

759 

1 

000 

1.000 

1 

000 

1.000 

Age  9  Bridge  -  Book  93 

1197 

0 

991 

C 

.054 

0 

556 

1 

000 

1. 000 

1 

GOO 

1.000 

Age  13  Bridge  -  Books  51-56 

5500 

1 

000 

0 

0u4 

0 

970 

1 

000 

1.000 

1 

000 

1.000 

Age  13  Bridge  -  Book  90 

1938 

0 

998 

0 

012 

0 

901 

1 

000 

1.000 

1 

000 

1.000 

Age  13  Bridge  -  Book  91 

1405 

0 

999 

0 

007 

0 

953 

1 

000 

1.000 

1 

000 

1.000 

Age  13  Bridge  -  Book  92 

1281 

1 

000 

0 

000 

1 

000 

1 

000 

1.000 

1 

000 

1.000 

Age  13  Bridge  -  Books  93 

1256 

0 

996 

0 

024 

0 

847 

1 

000 

1.000 

1 

000 

1.000 

Age  17  Bridge  -  Books  51-56 

4622 

0 

991 

0 

053 

0 

"71 

1 

000 

1.000 

1 

000 

1.000 

Age  17  Bridge  -  Books  61-67 

7052 

0 

992 

0 

050 

0 

654 

1 

000 

1.000 

1 

000 

1.000 

Age  17  Bridge  -  Book  90 

1786 

0 

993 

0 

.040 

0 

691 

1 

000 

1.000 

]. 

000 

1.000 

*  Number  includes  5,'^'    13-year-old  students  who  were  assessed  as  part  cf  che  International  Assessment  of 
Mathematics  and  Science  (s»ow  k  World  of  Differences:    An  International  Assessment  of  Mathematics  and  Science.  Tecbnis^al 
Report    [King,  Bertrand,  &  Dupuis,  1989]). 
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Table  B-7 

Distribution  of  Poststratif ication  Factors  -  Assessed  Students 


No.  of  Standard  25th  75th 

Sample  Cages  Mean        Deviation       Min.         Percentile       Median       Percentile  Max« 


Grade  V^ge  9 

Main 

23012 

1 

037 

0 

228 

0 

618 

0 

862 

1 

146 

1 

146 

1 

435 

Grade  A/Age  9 

Main  Winter 

12293 

0 

960 

0 

244 

0 

529 

0 

774 

0 

995 

1 

1 

857 

Grade  V^ge  9  Main  Spring 

10719 

1 

127 

0 

253 

0 

667 

0 

915 

1 

174 

1 

290 

1 

680 

Grade  8/Age  13 

Main 

36699* 

1 

046 

0 

165 

0 

715 

0 

939 

1 

051 

1 

\kl 

1 

381 

Grade  8/Age  13 

Main  Winter 

19131 

1 

007 

0 

271 

0 

664 

0 

833 

0 

924 

1 

146 

1 

680 

Grade  8/Age  13 

Main  Spring 

17568 

1 

083 

0 

224 

0 

752 

0 

958 

1 

027 

1 

173 

1 

556 

Grade  12/Age  17  Main 

32710 

1 

120 

0 

116 

0 

909 

1 

009 

1 

.110 

1 

243 

1 

380 

Grade  12/Age  17  Main  Winter 

18542 

1 

086 

0 

166 

0 

747 

0 

955 

1 

067 

1 

237 

1 

272 

Grade  12/Age  17  Main  Spring 

14168 

1 

152 

0 

118 

0 

943 

1 

061 

1 

156 

1 

251 

1 

591 

Age  9  Bridge  - 

Books  51-56 

5188 

1 

010 

0 

273 

0 

544 

0 

858 

1 

072 

1 

159 

1 

984 

Age  9  Bridge  - 

Book  91 

1274 

1 

025 

0 

190 

0 

656 

0 

992 

1 

048 

1 

118 

1 

262 

Age  9  Bridge  - 

Book  92 

1240 

1 

012 

0 

250 

0 

598 

0 

818 

1 

093 

1 

212 

1 

330 

Age  9  Bridge  - 

Book  93 

1197 

1 

084 

0 

236 

0 

577 

1 

024 

1 

051 

1 

127 

331 

Age  13  Bridge 

-  Books  51-56 

5500 

1 

072 

0 

167 

0 

926 

0 

970 

1 

030 

1 

110 

1 

548 

Age  13  Bridge 

-  Book  90 

1938 

1 

050 

0 

124 

0 

8^ 

0 

939 

1 

.061 

1 

111 

1 

252 

Age  13  Bridge 

-  Book  91 

1405 

1 

078 

0 

183 

0 

785 

0 

942 

1 

054 

1 

236 

1 

340 

Age  13  Bridge 

-  Book  92 

3  281 

1 

170 

0 

177 

0 

842 

0 

926 

1 

043 

1 

lei 

1 

310 

Age  13  Bridge 

-  Books  93 

1256 

1 

281 

0 

176 

0 

974 

1 

044 

1 

069 

1 

329 

1 

596 

Age  17  Bridge 

-  Books  51-56 

462? 

1 

259 

0 

376 

0 

939 

0 

997 

1 

098 

1 

327 

2 

210 

Age  17  Bridge 

-  Books  51  -  67 

7052 

1 

211 

0 

376 

0 

871 

1 

077 

1 

108 

1 

316 

2 

985 

Age  17  Bridge 

-  Book  90 

1786 

0 

993 

0 

.186 

0 

.985 

1 

087 

i 

195 

1 

282 

1 

625 

*  Number  includes  5,098  13-year-old  stu<i<^nts  who  were  assessed  as  part  of  the  International  Assessment  of 
Mathematics  and  Science  (see  A  World  of  Differences:    An  International  Assessmer.t  of  Mathematics  and  Science.  Technical 
Report    [King,  Bertrand,  6eDupuis,  1989]). 
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Table  B-8 


Distribution  of  Aggregate  Adjustments  to  Base  Weights  -  Assessed  Students 


No.  of 

Standard 

25  th 

75th 

Sa/nple 

Cases 

Mean 

Deviation 

Percentile 

Median 

Percentile 

Max, 

Grade  4/Age  9  Mair. 

23012 

1 

z45 

0 

A  ci 

.45? 

0 

CIA 

,  510 

1 

012 

1 

.  189 

1.393 

4.702 

Grade  8/Age  13  Main 

36699* 

I 

Q  ^  "7 

JO/ 

0 

.467 

0 

.  542 

1 

109 

1 

.240 

1.510 

4.757 

Grade  12/Age  17  Main 

32710 

1 

7  A  A 

u 

ICQ 
.  /DO 

U 

.  9/0 

1 

322 

1 

447 

1.793 

8.106 

Age  9  Bridge  -  Books  51-56 

5188 

1 

216 

0 

.401 

320 

0 

957 

1 

184 

1.347 

3.1''2 

Age  9  Bridge  -  Book  91 

1274 

1 

191 

0 

.330 

84 

1 

049 

1 

167 

1.320 

2Aj9 

Age  9  Bridge  -  Book  92 

1240 

^ 

222 

0 

.393 

U 

476 

1 

026 

1 

213 

1.389 

2.947 

Age  9  Bridge  -  Book  93 

1197 

1 

230 

0 

.437 

0 

377 

1 

052 

1 

161 

1.403 

3.645 

Age  13  Bridge  -  Books  51-56 

5500 

1 

220 

0 

.292 

0 

728 

1 

033 

1 

127 

1.300 

3.772 

Age  13  Bridge     Book  90 

1938 

1 

230 

0 

315 

0 

802 

1 

069 

1 

186 

1.315 

4.178 

Age  13  Bridge  -  Book  91 

1405 

1 

145 

0 

275 

0 

414 

0 

969 

1 

112 

1.293 

2.354 

Age  13  Blldge  -  Book  92 

1281 

1 

178 

0 

224 

0 

830 

0 

980 

1 

189 

1.332 

2.229 

Age  13  Bridge  -  Books  93 

1256 

1 

305 

0 

.404 

0 

496 

1 

089 

1 

257 

1.450 

3.685 

Age  17  Bridge  -  Books  51-56 

4622 

2 

027 

0 

956 

1 

021 

1 

266 

1 

639 

2.554 

6.708 

Age  17  Bridge  -  Books  61  -  67 

7052 

1 

951 

0 

993 

0 

930 

1 

274 

1 

608 

2.334 

9.134 

Age  17  Bridge  -  Book  90 

1786 

1 

937 

0 

778 

1 

110 

1.39A 

1 

592 

2.286 

5.495 

*  Number  Includes  5,098  13-year-old  students  who  were  assessed  as  part  of  the  International  ASi^essment  of 
Mathematics  and  Science  (see  A  World  of  Differences.    An  International  Assessment  of  Mathematics  and  Science.  Technical 
Report    [King,  Bertrand,  &  Dupuls,  1989]). 
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Table  B-9 


Distribution  of  Final  Student  Weight  -  Assessed  Students 


S£  le 

Grade  4/Age  9  Main 
Grade  8/Age  13  Main 
Grade  12/Age  17  Main 
Age  9  Bridge 
9  Bridge 
9  Bridge 
9  Bridge 
13  Bridge 
13  Bridge 
13  Bridge 
Bridge 
Bridge 
Bridge 
17  Bridge 


Age 
Age 
Age 
Age 
Age 
Age 

Age  13 
Age  13 
Age  17 
Age 

Age  17  Bridge  - 


56 


Books  51-56 
Book  91 
Book  92 
Book  93 
Books  51 
Book  90 
Book  91 
Book  92 
Books  93 
Books  51-56 
Books  61 
Book  90 


67 


JNo .  or 

Standard 

25th 

75th 

Cases 

Mean 

Deviation 

Percentile 

Median 

Percentile 

Max  t 

ZjUIZ 

1  OA 

1^4 . 

c 
D 

100. 

6 

J  /  , 

c 

,  0 

122. 

7 

170, 

,0 

261, 

.2 

131/, 

A 
►  H 

iio . 

vS5. 

1 

1  7 

7 

74. 

8 

104. 

.9 

149, 

.7 

1  A/iQ 

>  O 

O  O  "7  1  A 

TOO 

Izi . 

U 

71. 

4 

Q 1 
ji  . 

.  0 

68. 

0 

114. 

.2 

152, 

.5 

.  U 

00  J  . 

u 

518, 

.6 

1  QA 

■7 

.  / 

462. 

8 

727. 

.8 

1155, 

.3 

0  JO** , 

1 

1274 

r 
0 

1492. 

.7 

O  A  O 

242 , 

c 

.  0 

1369. 

7 

2222. 

.9 

3355, 

.1 

o 

>  J 

T  o  A  r\ 

o  c  o  ^ 
Zdoo  . 

/ 

1589. 

.8 

22/  , 

o 

.  0 

1381. 

4 

2163. 

.3 

3497, 

.3 

/  oy 0 , 

Q 

1197 

O  ^  O  1 

o 
0 

1654, 

.8 

A  7  O 

4/U , 

.  / 

1444. 

3 

2256. 

.7 

3788, 

.1 

1  JO^Z , 

>  0 

5500 

784. 

0 

349. 

.3 

174. 

.6 

549. 

7 

740. 

.0 

955, 

.1 

4352, 

.6 

1938 

1569. 

6 

610, 

.4 

326, 

.4 

1176. 

3 

1484. 

.6 

1961, 

.7 

6029, 

.8 

1405 

2170, 

4 

767, 

.0 

685. 

.2 

1612. 

6 

2163. 

.4 

2688, 

.5 

4436, 

.1 

1281 

2373, 

0 

1017. 

.8 

587, 

.5 

1859. 

2 

2265, 

.3 

2797. 

,9 

12106. 

.2 

1256 

2443, 

2 

1205. 

.5 

630, 

.3 

1821. 

3 

2328. 

.9 

2994, 

,6 

15380. 

.3 

4622 

971, 

0 

674. 

.2 

268. 

.5 

536. 

4 

802 

.8 

1152, 

.6 

5870, 

,1 

7052 

636. 

,4 

463. 

.8 

125, 

.6 

349. 

2 

531, 

,0 

744, 

.1 

6121, 

,6 

1786 

1925. 

,8 

1226. 

.2 

612, 

,4 

1106. 

6 

1566 

.0 

2251. 

.3 

8733, 

.1 

*  Number  Includes  5,098  13-year-old  students  who  were  assessed  as  part  of  the  International  Assessment  of 
Mathematics  and  Science  (see  A  World  of  Differences.    An  International  Assessment  of  Mathematics  and  Science.  Te<.hnical 
Report    (King,  Bertrand,  &  Dupuis,  1989]). 
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Table  B-IO 


Distributions  of  Weight  Components  for  Excluded  Studenr.s 

No.  of  Standard  25th  75th 

Component/Sample                       Cases  Mean  Deviation  Mln,  Percentile  Median       ferc^ntlle  Ma2L. 

Base  Weight 

Grade  4/Age  9                              2254  88.8  45.8  25.6  58.3  66.2            108.3  292.7 

Grade  8/Age  13                            3064  63.5  36.0  12.8  43.0  59.7              82.0  494.5 

Grade  U/Age  17                            538  174.7  83.8  78.7  107.2  140.6            228.7  388.1 

Grade  12/Age  17                           1527  70.5  51.3  25.0  41.8  50.3              98.1  330  9 


School  Nonresponse  Adjustment 
Grade  4/Age  9 
Grade  8/Age  13 
Grade  11/Age  17 
Grade  12/Age  17 

Session  Nonresponse  Adjustment 
Grav-'s  4/Age  9 
Grade  8/Age  13 
Grade  11/Age  17 
Grade  12/Age  17 


Grade  4/Age  9 
Grade  8/Age  13 
Grade  11/Age  17 
Grade  12/Age  17 

Student  Nonresponse  Adjustment 
Grade  4/Age  9 
Grade  8/Age  13 
Grade  11/Age  17 
Grade  12/Age  17 


2254 

1. 

070 

0. 

216 

0. 

796 

1. 

000 

1. 

000 

1.000 

3064 

1. 

119 

0. 

295 

0. 

760 

1. 

000 

1. 

000 

1.058 

538 

1. 

158 

0. 

338 

0. 

990 

1. 

000 

1. 

000 

1.078 

1527 

1. 

212 

0. 

393 

0. 

872 

1. 

000 

1. 

000 

1.260 

2254 

1. 

048 

0. 

153 

1. 

000 

1 

000 

1. 

000 

l.COO 

3064 

1. 

003 

0 

064 

1 

000 

1 

000 

1 

000 

1.000 

538 

1 

046 

0 

158 

1 

000 

1 

000 

1 

000 

1.000 

N/A 

us tment 

2254 

1 

0 

.066 

1 

000 

1 

.000 

1 

.000 

1.000 

3064 

1 

006 

0 

.089 

1 

.000 

1 

.000 

1 

.000 

1.000 

538 

JL 

.009 

0 

.109 

1 

.000 

1 

.000 

1 

.000 

1.000 

1527 

1 

.026 

0 

.179 

1 

.000 

1 

.000 

1 

.000 

1.000 

2254 

1 

.076 

0 

.168 

1 

.000 

1 

.000 

1 

.004 

1.040 

3064 

1 

.064 

0 

.188 

1 

.000 

1 

.000 

.000 

1.041 

538 

1 

.06 

0 

.114 

1 

.000 

1 

.000 

1 

.000 

1.056 

1527 

1 

.038 

0 

.117 

1 

.000 

1 

.000 

1 

.000 

1.032 

3.174 
2.847 
2.000 
3.045 


2.000 
1.513 
2.000 


2.039 
3.068 
2.270 
4.656 


1.882 
3.000 
1.444 
1.807 
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Table  B-10  (continued) 
Distributions  of  Weight  Components  for  Excluded  Students 


No.  of  Standard 

Component/Sairple  Cases  M<»an  Deviation 

Trimming  Factor 

Grade  A/Age  9  2253  0.936  0.179 

Grade  8/Age  13  3064  0.988  0.051 

Grade  11/Age  17  538  0.961  0.143 

Grade  12/Age  17  1527  0.972  0.131 

Poststratif ication  Factor 

Grade  4/Age  9  2253  0.981  0.271 

Grade  8/Age  13  3064  1.062  0.186 

Grade  11/Age  17  538  1.509  0.401 

Grade  12/Age  17  1527  1.109  0.130 

Combined  Weighted  Adjustments 

Grade  4/Age  9  2253  1.117  0.528 

Grade  8/Age  13  3064  1.256  0.461 

Grade  11/Age  17  538  1.814  0.766 

Grade  12/Age  17  1527  1.365  0.326 

Final  Student  Weight 

Grade  4/Age  9  2253  102.9  78.4 

Grade  8/.'^ge  13  3064              80.3  49.5 

Grade  11/Age  17  538  318.9  231.3 

Grade  12/Age  17  1527              88.0  47.8 


Min. 


0.301 
0.622 
0.363 
0.180 


0.618 
0.713 
0.912 
0.909 


0.314 
0.596 
0.912 
0.223 


25th 
Percentile 


1.000 
1.000 
1.000 
1.000 


0.696 
0.939 
1.104 
0.991 


0.696 
0.958 
1.280 
1.091 


Median 


1.000 
1.000 
1.000 
1.000 


1.069 
1.082 
1.548 
1.110 


1.107 
1.184 
1.692 
1.235 


75th 
Percentile 


1.000 
1.000 
1.000 
1.000 


1.164 
1.222 
1.805 
1.209 


1.349 
1.317 
2.019 
1.509 


18 

.3 

45 

1 

81 

.3 

139.1 

9 

.1 

52 

0 

70 

4 

96.4 

86 

9 

183 

2 

246 

5 

384.8 

23 

7 

51 

6 

71 

6 

115.3 

Max. 


1.000 
1.000 
1.000 
1.000 


1.447 
1.377 
2.465 
1.380 


4.282 
3.933 
6  547 
5.921 


566.4 
641,  2 
1889  ' 
333. 
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Table  B-11 


Distributions  of  Weight  Components  for  Teacher  Data  Files 


Component /Sa;nple 


No.  of 

Cases 


Mean 


Student  Nonresponse  Adjusted  Student;  Weight 


Grade  4 
Grade  8 


j9u1 
3570 


790.0 
869.8 


Standard 
Deviation 


403.2 
522.1 


Min^ 


251.7 
157.0 


25th 
Percentile 


D40.7 


75th 

Hedian  Percentile 


784.3 
762.4 


944.6 
1039.1 


Max. 


3004 . 2 
5344.7 


Teacher  Nonresponse  Adjustment 

Grade  4  3901  1.138  0.227  9.946  1.008  1.042 

Grade  8  3570  1.174  0.255  0.986  1.026  1.095 


Trimming  "'actor 
Grade  4 
Grade  8 

Poststratification  Factor 
Grade  4 
Grade  8 


3901 
3570 


3901 
3570 


Final  Teacher  Data  Student  Weight 
Grade  4  3901 
Grade  8  3570 


C.994 
0.997 


1.050 
1.021 


835.5 
873.1 


0.036 
0.023 


0.194 
0.111 


467.0 
488.3 


0.715 
0.757 


0.731 
0.844 


184.1 
155.5 


000 
000 


1.056 
0.967 


499.0 
557.3 


1.000 
1.000 


1.067 
0.981 


752.1 
772.3 


1.158 
1.219 


1.000 
1.000 


1 . 212 
1.133 


015.4 
1089.6 


2.319 
3.359 


1.000 
1.000 


1.415 
1.170 


3283.0 
5241.0 


539 


40 
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Contrast  Codings  and  Estimpted  Effects 
for  1988  NAEP  Conditioning  Varic^bles 
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34.1 


Appendix  G 


CONTRAST  CODINGS  AND  ESTIMATED  EFFECTS 
FOR  1988  NAEP  CONDITIONING  VARIABLES 


This  appendix  contains  information  about  the  conditioning  variables  ^'sed 
in  the  construction  of  plausible  values  for  Jhe  1988  r\ssessme..ts  of  reading, 
writing,  civics,  U.S.  history,  geography,  mathematics,  and  science. 

The  first  part  of  the  appendix  gives  the  contrast  codings  for  each  set 
of  conditioning  variables  used  in  1988.     Codings  for  the  common  conditioning 
variables,  which  were  used  for  cross-sectional  studies  in  reading,  writing, 
civicii,  U.S.  history,  aad  geography,  are  given  In  Table  C-1.     In  addition, 
subject-specific  conditioning  variables  used  in  each  cioss-sectlonal  study  are 
given  for' reading  (Table  C-2),  v/riting  (C-4) ,  civics  (C-6),  U.S.  history 
(C-8),  and  geography  (C-9).     The  complete  set  of  conditioning  variables  for 
each  subject  area  in  which  plausible  values  were  constructed  for  tre-^.d  studi  s 
are  given  for  reading  (Table  C-3),  writing  (C-5),  civics  (C-7),  mathematics 
(C-10),  and  science  (C-11). 

The  second  part  of  the  appendix  shows  the  estimated  effects,  by  subject 
area,  sample (s),  and  age  class,  for  the  conditioning  variables  used  in  cross- 
sectional  studies  for  reading  (Tables  C-12  to  C-18),  civics  (C-22  to  C-27), 
U.S.  history  (C-34  to  C-36),  and  geography  (C-37)  and  for  the  conditioning 
variables  used  in  trend  studies  for  reading  (C-19  to  C-21),  civics  (C-28  to 
C-33),  mathematics  (C-38  to  C-40) ,  and  science  (C-41  to  C-43). 

Note  that  all  effect  estimates  are  in  the  metrics  used  in  the  original 
calibration  of  the  scale.    The  transformations  needed  to  represent  these 
effects  in  terms  of  the  metric  of  the  final  reporting  scales  appear  in  the 
chapters  that  describe  the  scaling  of  each  subject  area.    Note  also  that 
certain  conditioning  variables  do  not  have  effect  estimates.     This  is  because 
those  variables  are  approximate  linear  combinations  of  the  other  conditioning 
variables. 

Some  conditionfnp  variables  were  constructed  by  re::oding  the  values  of  a 
data  variable  or  by  combi.*ing  and  receding  data  from  two  or  more  variables.  A 
descriptica  of  how  these  conditioning  variables  were  derived  for  each  subject 
area  is  provided  in  Appendix  D. 


499 


Tables  C-1  through  C-11 
CONTRAST  CODINGS  FOR  1988  NAEP  CONDITIONING  VARIABLES 
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Table  C-1 

Contrast  Codings  for  1988  Conmion  Conditioning  Variables 


Conditioning  Age  Contrast 

Variable  Classes  Variable  Coding  Coding* 


Overall 

All 

~  *"  " 

1 

Gender 

All 

1  Male 

0 

2  Female 

1 

Ethnicity 

All 

1  White 

000 

2  Black 

100 

3  Hispanic 

010 

4  Asian  American 

001 

5  American  Indian 

000 

6  Unclassified 

000 

BLK  Missing 

000 

STOC 

All 

2  Low  Metro 

00 

3  High  Metro 

10 

1,4-7  All  Others  and  Missing 

01 

Region 

All 

All 

i  Northeast 

uuu 

^  o  oucn  e  as  c 

1  on 

3  Central 

010 

4  West 

001 

Parents '  Educ  ation 

Ai: 

1  <  High  School 

0000 

2  High  School  Grad 

1000 

3  Post-High  School 

0100 

4  College  Grad 

0010 

BLK  Missing  and  I  T  a't  Know 

0001 

Items  in  the  Home 

All 

1  0  to  2  of  the  four  i::ems 

00 

(Items  asked  about  are 

2  Three  of  the  four  items 

10 

newspaper,  >25  books, 

3  Four       the  four  itenis 

01 

er/:lopedia,  and  magazines 

BLK  Missing 

00 

Two  or  more  missing  -  Missing.) 


^  Multicoluran  entries  without  overbars  indicate  multiple  contrasts. 
"Items  in  the  home,"  for  example,  induces  two  contracts.  A  response  of  2  vs. 
all  other  responses,  and  a  response  of  3  v£.  all  other  responses.  Barred 
columns  treated  as  one  contrast. 
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Table  C-1  (continued) 


Contrast  Codin;^s  for  1988  Conmion  Conditioning  Variables 


Conditioning  Ag^  Contrast 

Variable  Classes  Variable  Coding  Coding* 

TV  Watching  All  1  None  0  00 

2  One  hour  or  less  per  day  1  01 

3  Two  hours  2  04 

4  Three  hours  3  09 

5  Four  hours  4  16 

6  Five  hours  5  25 

7  Six  or  more  hours  per  day  6  36 
BLK  Missing  3  09 

Home  Language  Minority         All  1  Never  0 

(How  often  do  people  in  2,3  Sometimes,  Always  1 

your  home  speak  a  language  BLK  ^iissing  0 

other  than  English) 

Homework  9  1  Don't  have  any  100 

2  Don't  do  any  010 

3  1/2  hour  Oil 

4  One  hour  012 

5  >  One  houi  013 
BLK  Missing  000 


13,  17  1  Don't  have  any  100 

2  Don't  do  any  010 

3  1/2  hour  Oil 

4  One  hour  012 

5  Two  hours  013 

6  >  Two  hours  014 
BU^  Missing  000 

Percent  in  Lunch  Program     All  0  000  0 

1  001  0 

2  002  0 

99  099  0 

100  100  0 
BLK  000  1 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-1  (continued) 
Contrast  Codings  for  1988  Common  Conditioning  Variables 


Conditioning  Age  Contrast 

Variable  Classes  Variable  Coding  Coding* 

Percent  White  in  School       All  0  -  49  White  Minority  10 

50  -  79  Integrated  01 
80  -  100  Predominantly  White  00 
BLK  Missing  00 

Age  by  Grade  All  1  <  Modal  Age,  Modal  Grade  0000 

2  Modal  Age,  <  Modal  Grade  1000 

3  Modal  Age,  Modal  Grade,  and 
Missing  0100 

4  Modal  Age,  >  Modal  Grade**  0010 

5  >  Modal  Age,  Modal  Grade  0001 


Public  V.  Private  Schools  All 


Someone  at  Home  Helps 
with  Homework 


Went  to  Preschool 


Sir^ie/Multipie  Pare.it 
at  Home 


All 


1  Public 

2  Private 

3  Catholic 

4  Bureau  of  Indian  Affairs 

5  Dept.  of  Defense 
BLK  Missing 

1  Almost  Every  Day 

2  Once  or  Twice  a  Week 

3  Once  or  Twice  a  Month 

4  Never 

5  Don't  Have  Homework 
BLK  Missing 

1  Yes 

2  No 

3  I  Don't  Know 
BLK  Missing 

1  Yes  to  Father  and  Mother 
at  Home 

2  Any  Other  Responses 
BLK  Missing 


0 
1 
1 
1 
1 
1 

1 
1 

0 
0 
0 
0 

1 

0 
0 
0 


0 
0 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  colximns  treated  as  on^i  contrast. 

**  Category  not  applicable  for  age  class  17. 
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Table  C-1  (continued) 


Contrast  Codings  for  1988  Common  Conditioning  Variables 


Conditioning  Age  Contrast 

Variable  Classes  Variable  Coding  Coding* 


Mother  at  Home 

All 

1  Yes 

1 

(Does  your  mother 

2  No 

0 

live  at  home) 

dus,  fiissing 

A 
U 

Mother  Works  Outside 

9 

1  Yes 

1 

of  Home 

2  No 

0 

J  Motner  Not  at  Home 

A 

u 

BLK  Missing 

U 

13,  17 

1  Yes,  Full -Time 

1 

2  Yes,  Part-Time 

1 

3  No 

0 

4  Mother  Not  at  Home 

0 

BLK  Missing 

A 

u 

Grownup  at  Home 

9 

1  Yes 

1 

Right  After  School 

0  No 

0 

i>LJ\  nissing 

A 
U 

Pages  a  Day  Read  for 

All 
All 

1  nore  inan 

1  1 

School  and  Homework 

i  10  -  zu 

XJL 

'^11       1  '^ 
J    11    -  1j 

1  1 

4  6-10 

10 

5    5  or  Fever 

00 

BLK  Missing 

00 

Do  You  Expect  to 

13 

1  Yes 

1 

Graduate  from  High  School 

2  No 

0 

BLK  Missing 

0 

Days  of  School  Missed 

13,17 

1  None 

1 

Last  Month 

2  1  or  2  Days 

1 

3  3  or  4  Days 

0 

4  5  to  10  Days 

0 

5  More  ^han  10  Days 

0 

BLK  Missing 

0 

*  Multicoluran  entrJ.es  without  overbars  indicate  niu?tiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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5.17 


Table  C-1  (continued) 
Contrast  Codings  for  1988  Common  Conditioning  Variables 


Con'"-..tioning                        Age  Contrast 

Variable                          Classes                 Variable  Codf.n^  Coding* 

Grades  in  School                   13.  17           1  A  4.0 

2  A-B  3.5 

3  B  3.0 

4  B-C  2.5 

5  C  2.0 

6  C-D  1.5 

7  D  1.0 

8  <  D  0.5 
BLK  Missing  2.0 

High  School  Program              17                 1  General  00 

2  College  Preparatory  10 

3  Vocational,  Technical  Oi 
BLK  Missing  00 

Post -secondary  Plans            17                 1  Work  Full  Time  00 

2  Two-year  College  10 

3  Four-year  College  01 

4  ether  00 
BLK  Missing  00 

Hours  of  Outside  Work          17                 1  None  0 

2  <  6  Hours  4 

3  6  to  10  Hours  8 

4  11  to  15  Hours  13 

5  16  to  20  Hours  18 

6  21  to  25  Hours  23 

7  26  to  30  Hours  28 

8  >  30  Hours  40 
BLK  Missing  0 

Type  of  English  Class  In      17                 1  Not  Taking  00 

2  Advanced  Placement  10 

3  College  Preparatory  10 

4  General  00 

5  Remedial  01 
BLK  Missing  00 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-2 

Contrast  Codings  for  1988  Reading  Cross-sectional  Conditioning  Variables 


Conditioning  Age         Variable  Variable  Contrasr 

y     '.able                         Classes         Name                     Coding  Coding* 

Home  cnvlronnent                   All         EUiOME               Low  1 

and  support                                                               Meaiuin  2 

High  3 

Students'  independent          All         RINDRDG            Low  1 

reading                                                                    Medium  2 

High  3 

Students'  use  of                   13,  17    RUSEMAT            Low  1 

materials                                                                  Medium  2 

High  3 

Teacher  Instructional           13,  17    RTEACH              Low  1 

behaviors                                                                 Medluir  2 

High  3 

Students'  behaviors              17           RBEHAV              Low  1 

and  support                                                               Medium  2 

High  3 

Reading  strategies                17           RSTRATG            Low  1 

Medium  2 

High  3 

Students'  school  and            17           RSCHWRK            Low  1 

coursework                                                               Medium  2 

High  3 

Magazines  at  home                 9            R800101            Yes  1 

No  2 

Kxsslng  2 

Books  at  home                       All         R800301            10  or  fewer  1 

11  to  20  2 

21  to  30  3 

More  than  30  4 

Missing  1 


*  Multlcolumn  entries  without  overbars  Indicate  multiple  contrasts. 
Barred  coltnnns  treated  as  one  contrast. 
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Table  C-2  ''continued) 


Contrast  Codings  for  1988  Reading  Cross -sectional  Conditioning  Variables 


Conditioning  Age         Variable  Variable  Contrast 

Variable                          Classe;.         Name                     Codin;?,  Codinfc,^ 

Read    Jtories                          All         R800401      Alm'^^st  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

Never  or  hardly  ever  4 

I  don't  know  4 

Missing  4 

Read  aloud                             9            R800501     Almost  every  day  1 

Once  or  cwice  a  week  2 

Once  or  twice  a  moiich  3 

Never  or  hardly  ever  4 

Missing  4 

Read  for  fuu                         All         S003501     Almost  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

A  few  times  a  year  4 

Never  or  hardly  ever  5 

Missing            "  5 

Tell  about  book                    All         S003502     Almost  evtry  day  1 

Oncve  or  twice  a  week  2 

0     „        twice  a  month  3 

-  times  a  year  4 

Never  or  hardly  ever  5 

Missing  5 

Books  from  library               All         S003503     Almost  eery  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

A  few  times      *'ear  4 

Never  or  ha:   iy  ever  5 

Missing  5 


*  Multicolunn  entries  wirhout  overbars  indicate  multiple  oorttas.s. 
Barred  columns  treated  as  one  contrast. 
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Table  C-2  (continued) 


Contrast  Codings  for  1988  Reading  Cross -sectional  Conditioning  Variables 


Conditioning  Age        Variable  Variable  Contrast 

Variable                         Classes         Name                     Codinjg  Coding* 

Buy  own  books                       All         S003504     Almost  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

A  few  times  a  year  4 

Never  or  hardly  ever  5 

Missing  5 

Kind  of  reader                      All         S003301      A  poor  reader  1 

A  good  reader  2 

A  very  good  reader  3 

I  don' t  know  1 

Missing  1 

Teacher  asks                         9            R800701      Almost  every  day  1 

Once  or  twic3  a  week  2 

Once  or  twice  a  month  3 

Never  or  hardly  ever  4 

Missing  4 

Magazines  at  home                 13,  17    R800201      Yes  1 

No  2 

Missing  2 

Talk  about  reading               13,  17    R800601      Almost  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  t  month  3 

Never  or  hardly  ever  4 

Missing  4 

Read  after  TV  show               13,  17    S003505      Almost  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

A  few  times  a  year  4 

Never  or  hardly  ever  5 

Missing  5 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 


Table  C-2  (continued) 
Contrast  Codings  for  1988  Reading  Cross -sectional  Conditioning  Variables 


Conditioning  Age         N^ariable  Variable  Contrast 

Variable                          Classes         Name                     Coding  Coding* 

Author  you  like                    13,  17    S003506     Almost  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

A  few  times  a  year  4 

Never  or  hardly  ever  5 

Missing  5 

Read  in  spare  time                13,  17    L801101     Never  1 

Fiction  2 

Nonfiction  2 

Fiction  and  nonfiction  2 

Missing  1 

Use  dictionary                      13,  17    R800801     Almost  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

A  few  times  a  year  4 

Never  or  hardly  ever  5 

Missing  5 

Use  encyclopedia                   13,  17    S007309      Almost  e-^ery  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

A  few  times  a  year  4 

Never  or  hardly  ever  5 

Missing  5 

Analyze  reading                    13,  17    R801201     A  lot  1 

Some  2 

Not  at  all  3 

Missing  3 

Write  on  reading                  13,  17    S008501     Almost  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

A  few  times  a  year  4 

Never  or  hardly  ever  5 

Missing  5 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  colurons  rreated  as  one  contrast. 
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Table  C-2  (continued) 


Contrast  Codings  for  1988  Reading  Cross-sectional  Conditioning  Variables 


Conditioning                        Age         Variable               Variable  Contrast 

Variable                          Classes         Name                     Coding  Coding* 

Teacher-main  idea                 13,  17    S004602     Almost  every  time  1 

More  than  1/2  the  time  2 

About  1/2  the  time  3 

Less  than  1/2  the  time  4 

Never  or  hardly  ever  5 

Missing  5 

Teacher-vocabulary                13,  17    S004601     Almost  every  time  1 

More  than  1/2  the  time  2 

About  1/2  the  time  3 

Less  than  i/2  the  time  4 

Never  or  hardly  ever  5 
Missing 

Teacher-questions                 13,  17    S00*'>701     Almost  every  time  1 

More  than  1/2  the  time  2 

About  1/2  the  time  3 

Less  than  1/2  the  time  4 

Never  or  hardly  ever  5 

Hissing  5 

Can't  understand                   13,  17    S008601     A  lot  1 

Some  2 

None  3 

Missing  1 

Read  story,  novel                 17           S004301     Almost  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

A  few  times  a  year  4 

Never  or  hardly  ever  5 

Missing  5 

Read  newspaper                      17           S004304     Almost  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  month  3 

A  few  times  a  year  4 

Never  or  hardly  ever  5 

Missing  : 


*  Multicolumn  entries  without  overbars  ir^dicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-2  (continued) 
Contrast  Codings  for  1988  Reading  Cross-sectional  Conditioning  Variables 


Conditioning  Age         Variable  Variable  Contrast 

Variable                          Classes         Name                     Coding  Coding* 

Read  magazine                        17           S004305      Almost  every  day  1 

Once  or  twice  a  week  2 

Once  or  twice  a  laonth  3 

A  few  times  a  year  4 

Never  or  hardly  ever  5 

Missing  5 

Read  material  over                17           S005101     Almost  every  time  1 

More  than  1/2  the  time  2 
About  1/2  the  time 

Less  than  1/2  the  time  4 

Never  or  hardly  ever  5 

Missing  5 

Take  notes                             17           S005102     Almost  every  time  1 

More  than  1/2  the  time  2 

About  1/2  the  time  3 

Less  than  1/2  the  time  4 

Never  or  hardly  ever  5 

Missing  5 

Make  outlines                        17           S005103      Almost  every  t  ne  1 

More  than  1/2  the  time  2 

About  1/2  the  time  3 

Less  than  1/2  the  time  4 

Never  or  harily  ever  5 

Missing  5 

Answer  questions                   17           S005104     Almost  every  time  1 

More  than  1/2  the  time  2 

About  1/2  the  time  3 

Less  than  1/2  the  time  ^ 

Never  or  hardly  ever  5 

Missing  5 

AP  English                             17           S006403      Yes  1 

No  2 

Missing  2 


*  Multicolumn  entries  without  overbars  indicate  multiple  contirasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-2  (continued) 


Contrast  Codings  for  1988  Reading  Cross-sectional  Conditioning  Variables 


Conditioning 
Variable 

Remedial  English 


Time  on  literature 


Age 
Classes 

17 


17 


Variable 
Name 

S006601 


R800901 


Time  on  homework 


17 


R801001 


English  grades 


17 


R801101 


Reading  level 
of  student 


Grd.  4  T019701 


Variable 
Coding 


Contrast 
Coding^ 


Yes 

1 

No 

2 

Missing 

1 

Most  of  tht  time 

1 

More  than  1/2  the  time 

2 

About  1/2  the  time 

3 
•J 

Less  than  1/2  the  ^imu 

4 

None  or  almost  none 

5 

Missing 

5 

None  assigned 

0 

Don't  do  it 

0 

Less  than  1  hour 

1 

1  hour 

2 

2  i*ours 

3 

3  hours 

4 

4  hours 

5 

5  or  more  hours 

6 

Missing 

0 

Mostly  A 

1 

A  and  B 

2 

Mostly  B 

3 

B  and  C 

4 

Mostly  C 

5 

C  and  D 

6 

Mostly  D 

7 

Mostly  below  D 

8 

Missing 

8 

1  above  grade  level 

1 

2  at  grade  level 

2 

3  below  grade  level 

3 

Missing 

3 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts 
Barred  columns  treated  as  one  contrast. 


Table  C-2  (continued) 


Contrast  Codings  for  1988  Reading  Cross-sectional  Conditioning  Variables 


Conditioning 

Age 

Variable 

Variable 

oontrasc 

Variable  Classes 

Name 

uoaing 

Coding* 

Level  of  student's 

Grd. 

4 

T020001 

1  high 

11 

reading  group 

2  average 

IZ 

J  low 

4  do  not  have  groups 

UU 

Missing 

00 

Help  student  with  reading 

Grd. 

4 

TOzOlll 

1  almost  every  day 

1 

with  reading  aloud  aloud 

2  several  times  a  week 

Z 

3  once  a  week  or  less 

i 

Missing 

1 

Help  student  with 

Grd. 

4 

iUZUiZi 

1  almost  every  day 

1 

comprehensive  skills 

2  several  times  a  week 

Z 

3  once  a  week  or  less 

Q 

J 

Missing 

1 

nexp  scuoenc  wicn  woro- 

A 
*+ 

iUzUi Ji 

i  aimosc  every  oay 

1 

1 

attack  skills 

z     e vcL  Jix,  L. j.nies  a  Wc ctv 

0 

3  once  a  week  or  less 

3 

Missing 

1 

Help  student  with 

Grd. 

4 

i  aimosc  every  aay 

1 

X 

vocabulary 

2  several  times  a  week 

2 

3  once  a  week  or  less 

3 

n  J.d  d  i-Ilg 

1 

Does  student  receive 

Grd. 

4 

iUZU /Ul 

1  yes 

1 

remedial  reading  prog 

z  nc 

0 

Mi  ^ ^  1  ntr 

I 

Are  students  assigned  to 

Grd. 

4 

T012601 

1  yes 

1 

class  by  ability 

2  no 

2 

Missing 

2 

Reading  ability  level 

Grd. 

4 

T021701 

1  mostly  high 

1 

of  student  in  class 

2  mostly  average 

2 

3  mostly  low 

3 

4  mixed  ability  levels 

2 

Missing 

2 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-2  (continued) 


Contrast  Codings  for  1988  Reading  Cross- sectional  Conditioning  Variables 


Conditioning 

Age 

Variable 

Variable 

Contrast 

Variable  Classes 

Name 

Coding 

Coding* 

Time  spent  on  typical  day 

Grd. 

4 

T021801 

1  30  minutes  or  less 

1 

day  for  direct  instruction 

2  60  minutes 

2 

3  90  minutes 

3 

4  IzO  minutes  or  more 

3 

Missing 

3 

Percent  of  reading 

Grd. 

4 

T021901 

<  10 

11 

instruction- individual 

10-14 

12 

15-24 

13 

25-100 

14 

Hissing 

00 

Percent  of  reading 

Grd. 

4 

T021902 

0-24 

11 

instruction-small  group 

25-49 

12 

50-74 

13 

75-100 

14 

Missing 

00 

Percent  of  reading 

Grd. 

4 

T021903 

0-24 

11 

instructi'jn-whole  class 

25-49 

12 

50-74 

13 

75-100 

14 

Missing 

00 

Are  students  assigned 

Grd. 

4 

1  yss 

1 

to  groups  by  ability 

z  no 

2 

•J    L/U    IIU  L.  loim 

reading  groups 

3 

Missing 

3 

Use  same  reading  bock 

Grd. 

4 

T022201 

1  yes 

100 

for  all  students  in  class 

2  No,  diff  levels 

within  same  basal 

010 

3  No,  use  different 

basal  series 

001 

4  No,  use  other  books 

or  magazines 

000 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-2  (continued) 


Contrast  Codings  for  1988  Reading  Cross-sectional  Conditioning  Variables 


Conditioning  Age         Variable  Variable  Contrast 

Variable  Classes         Name  Coding  Coding^ 


Number  of  resources  used 

Grd. 

4 

T022301- 

0 

0 

(of  5--book  collection, 

T022J05 

1 

1 

1 

children's  paper, 

2 

2 

reading  kits,  computer. 

3 

3 

instruction  software 

4 

4 

5 

5 

How  often  -  read  aloud  to 

Grd. 

4 

T022421 

1 

almost  every  day 

1 

students 

2 

1-2  times  a  week 

2 

3 

1-2  times  a  month 

o 

J 

4 

a  few  times  a  year 

4 

5 

never 

c 

5 

Missing 

5 

How  often  -  have  students 

Grd. 

4 

T022431 

1 

almost  every  day 

1 

meet  in  small  groups 

2 

1-2  times  a  week 

2 

3 

1-2  times  a  month 

3 

4 

a  few  times  a  year 

4 

5 

never 

C 

3 

Missing 

5 

How  often  -  have  students 

Grd. 

4 

T022441 

1 

almost  every  da^ 

1 

write  about  something 

2 

1-2  times  a  week 

2 

they  read 

3 

1-2  time  3  a  month 

3 

4 

a  few  times  a  year 

4 

5 

never 

5 

Missing 

5 

How  often  -  have  students 

Grd. 

4 

T022451 

1 

almost  every  day 

1 

complete  workbooks 

2 

1-2  times  a  week 

2 

3 

1-2  times  a  month 

3 

4 

a  few  times  a  year 

4 

5 

never 

5 

Missing 

5 

*  Multicolvunn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-2  (continued) 
Contrast  Codings  for  1988  Reading  Cross -sectional  Conditioning  Variables 


Conditioning  Age         Variable  Variable  Contrast 

Variable                          Classes         Name                     Coding  Coding^- 

How  often  -  have  students    Grd.  4    T022461      1  almost  every  day  1 

rer  i  books  they  choose                                      2  1-2  times  a  week  2 

3  1-2  times  a  month  3 

4  a  few  times  a  year  4 

5  never  5 
Missing  5 

How  often  -  have  students    Grd.  4    T022471      1  almost  every  day  1 

read  information  material                                 2  1-2  times  a  week  2 

3  1-2  times  a  month  3 

4  a  few  times  a  year  4 

5  never  5 
Missing  5 

How  often  -  take  class  to    Grd.  4    T022481      1  almost  every  day  1 

the  library                                                       2  1-2  times  a  week  2 

3  1-2  tima^i  a  month  3 

4  a  few  times  a  year  4 

5  never  5 
Missing  5 

Highest  academic  degree       Grd.  4    T0232O1      1  High  school  diploma  1 
held                                                                    2  Associate  degree/ 

voc  certificate  2 

3  Bachelor's  degree  2 

4  Master's  degree  3 

5  Educatnl  specialist  4 

6  Doctorate  4 

7  First  professional 
degree  4 

Missing  2 

Number  of  special                 Grd.  4    T023502-    0  0 

trainings  in  teaching                        T023505      1  1 

reading  (of  4--inservice,                                  2  2 

undergrad,  graduate,                                          3  3 

continuing  educ.)                                              4  4 


*  Multicoluran  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-2  (continued) 


Contrast  Codings  for  1988  Reading  Cross-sectional  Conditioning  Variables 


Conditioning  Age        Variable  Variable  Contrast 

Variable                         Classes         Name                    Coding  CodlnR* 

Ability  to  get  desired        Grd.  4    T024001      1  get  all  needed  1 

instructional  materials                                   2  get  most  needed  2 

and  resources                                                    3  get  some  needed  3 

H  don't  get  any  needed  3 

Missing  3 

If  could  start  over,            Grd.  4    T024101     1  certainly  1 

would  become  a  teacher                                     2  probably  2 

3  chances  about  even  3 

4  probably  not  4 

5  certainly  not  5 
Missing  3 


*  Multicolumn  entries  witho  t  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-3 


Contrast  Codings  for  1988  Reading  Trend  Conditioning  Variables 


Conditioning  Contrast 
Variable  Ages  Variable  Coding  Coding* 


Overall 

kV 

1 

Gender 

All 

1  Male 

0 

2  Female 

1 

Ethnicity 

All 

1  White 

000 

2  Black 

100 

3  Hispanic 

010 

4  Asian  American 

001 

5  American  Indian 

000 

6  Unclassified 

000 

STOC 

All 

2  Low  Metro 

00 

3  High  Metro 

10 

1,4-7  All  Others  and  Missing 

01 

Region 

All 

1  Northeast 

000 

2  Southeast 

100 

3  Central 

010 

4  West 

001 

Parents'  Education 

All 

1  <  High  School 

0000 

2  High  School  Grad 

1000 

3  Post-High  School 

0100 

4  College  Grad 

0010 

BLK  Missing  and  I  Don't  Know 

0001 

Items  in  the 

Home 

All 

0  None  of  the  six  items 

00 

(the  items  asked  are: 

1  One  of  the  six  items 

10 

Newspaper, 

Dictionary, 

2  Two  of  the  six  items 

20 

>  25  books, 

Encyclopedia, 

3  Three  jf  the  six  items 

30 

Video  games 

,  Magazines) 

4  Four  of  the  six  items 

40 

5  Five  of  the  six  items 

50 

6  .Jix  of  the  six  items 

60 

.^LK  Missing 

01 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-3  (continued) 


Contrast  Codings  for  1988  Reading  Trend  Conditioning  Variables 


Conditioning 
Variable 

TV  Watching 


Ages 
All 


Hontbwork 


All 


Home  Language  Minority  All 


Pages  Read 


All 


Percent  in  Lunch  Program  All 


Variable  Coding 

1  None 

2  One  hour  or  less  per  day 

3  Two  hours 

4  Three  hours 

5  Four  hours 

6  Five  hours 

7  Six  or  mere  hours  per  day 
BLK  Hissing 

1  Don' t  have  any 

2  Don't  do  any 

3  <  one  hour 

4  One  to  two  hours 

5  >  two  hours 
BLK  Missing 

1  English 

2  Spanish 

3  Other 
BLK  Missing 

1  More  than  20 

2  16-20 

3  11-15 

4  6-10 

5  5  or  fewer 
BLK  Missing 

0 
1 
2 


Contrast 
Coding* 

0  00 


01 
04 
09 
16 
25 
36 
09 


1 

2 
3 
14 
5 
6 
3 

00 
00 
10 
20 
30 
01 

00 
10 
10 
0] 

10 
10 
10 
10 
00 
01 


0000 
0010 
0020 


99 
100 

BLK  Missing 


0990 
1000 
0001 


*  Multicolupin  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-3  (continued) 


Contrast  Codings  for  1988  Reading  Trend  Conditioning  Variables 


Conditioning 

Contrast 

Variable 

Aees 

Variable  Codinp 

Coding* 

Percent  White  in  School 

All 

0-49     White  Minority 

100 

50-79  Integrated 

010 

80-100  Predominantly  White 

001 

BLK  Missing 

000 

Courses  Taken 

9,13 

0  None  of  the  seven 

00.00 

(The  courses  asked  about 

1  One  of  the  seven 

01.00 

are  Gym,  Art,  Music, 

2  Two  of  the  seven 

02.00 

Foreign  Lang,  Computer, 

3  Three  of  the  seven 

03.00 

Drama,  Science. 

4  Four  of  the  seven 

04.00 

3  or  more  missing  — 

5  Five  of  the  seven 

05.00 

missing) 

6  Six  of  the  seven 

06.00 

7  All  seven 

07.00 

BUC  Missing 

00.01 

Age  by  Grade 

All 

1    <  modal  age.  At  modal  grade 

0000 

2  At  modal  age,    <  modal  grade 

1000 

3  At  modal  age.  At  modal  grade 

0100 

4  At  modal  age,    >  modal  grade 

0010 

5    >  modal  age.  At  modal  grade 

0001 

BLK  Missing 

0100 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 


Table  C-4 


Contrast  Codings  for  1988  Writing  Cross -sectional  Conditioning  Variables 


Conditioning 
Variable 

Enjoy  Writing 


Grades 
All 


Positive  Teacher 
Feedback 


All 


Negative  Teacher 
Feedbiick 


Amount  Written  for 
English  Class 


All 


8,12 


Amount  Written  for 
History  Class 


8,12 


Contrast 

Variable  Coding  Coding* 

1.  Almost  Always  5 

2.  More  than  half  the  time  4 

3.  Half  the  time  3 

4.  Less  than  half  the  time  2 

5 .  never  1 
BLK  missing  3 

0  No  positive  feedback  0 

1  Little  positive  fe'  ""back  1 

2  Some  positive  feedback  2 

3  A  Lot  of  positive  feedback  3 
BIK  Kissing  0 

0  no  negative  feedback  0 

1  Little  negative  feedback  1 

2  Lots  of  negative  feedback  2 
BLK  missing  0 

alot  of  short,  never  long  100000 

a  lot  of  short,  a  lot  of  long  010000 

a  lot  of  short,  some  long  001000 

some  short,  some  long  OOOlOO 

some  short,  never  long  000010 

some  short,  a  lot  of  long  000001 

never  short,  a  lot  of  long  000001 

never  short,  some  long  000001 

never  short,  never  long  000000 

either  missing  000100 

a  lot  of  short,  never  long  100000 

a  lot  of  short,  a  lot  of  long  010000 

a  lot  of  short,  some  long  001000 

some  short,  some  long  000100 

some  short,  never  long  000010 

some  short,  a  lot  of  long  000001 

never  short,  a  lot  of  long  000001 

never  rhort,  some  long  000001 

never  short,  never  long  000000 

either  missing  000100 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-4  (continued) 


Contrast  Codings  for  1988  Writing  Cross -sectional  Conditioning  Variables 


Conditioning  Contrast 
Variable  Grades  Variable  Coding  Coding* 

Instinaction  on  8,12  1.  Almost  Always  S 

Writing  Process  II  2.  More  than  half  the  time  4 

3.  Half  time  3 

4.  Less  than  half  the  time  2 

5.  Never  1 
BLK  missing  3 


Success  in  English 

8,12 

1.  a  lot 

1 

Class  II 

2 .  some 

2 

3 .  none 

3 

BLK  missing 

2 

Success  in  English 

12 

1  Mostly  A 

8 

Class  111 

2  Between  A  and 

B 

7 

3  Mostly  B 

6 

4  Between  B  and 

C 

5 

5  Mostly  C 

4 

6  Between  C  and 

D 

3 

7  Mostly  D 

2 

8  Mostly  <  D 

1 

BLK  missing 

5 

Time  Spent  on 

12 

1  lots  of  time 

4 

Writing  III 

2  some  time 

3 

3  little  time 

2 

4  no  time 

1 

BLK  missing 

2 

Revisions                              12                 1  Almost  Always  5 

2  More  than  half  the  time  4 

3  Half  the  time  J 

4  Less  than  half  the  time  2 

5  Never  1 
BLK  Missing  3 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasti>. 
Barred  columns  treated  as  one  contrast. 
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Table  C-4  (continued) 


Contrast  Codings  for  1988  Writing  Cross -sectional  Conditioning  Variables 

Conditioning  Contrast 

Variable                         Grades                   Variable  Coding:  Codiiig* 

Planning                                15.                 1  Almost  Always  5 

2  More  than  half  the  time  4 

3  Half  the  time  3 

4  Less  than  half  the  time  2 

5  Never  1 
BIK  missing  3 

Time  Spent  on  Writing  II       8                 1  Weekly  4 

2  Monthly  3 

3  Yearly  2 

4  Never  1 
BLK  Missing  3 

Time  Spent  on  Writing  14                 1  Weekly  100 

2  Monthly  010 

3  Yearly  cOl 

4  never  000 
BLK  missing  010 

Success  in  English                 4                 1  Daily  1000 

Class  I                                                      2  Weekly  0100 

3  Monthly  0010 

4  Yearly  0001 

5  Never  0000 
BLK  Missing  0010 

Instruction  on                       4                 1  Almost  Always  5 

Writing  Process  I                                    2  More  than  half  the  time  4 

3  Half  time  3 

2  l^ss  than  half  tne  time  2 

1  Never  1 

BLK  Missing  3 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  coltunns  treated  as  one  contrast. 
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Table  C-5 


Contrast  Codings  for  1988  Writing  Trend  Conditioning  Variables 


Condi  tioning  Con  t r as  t 

Variable  Ages  Variable  Coding  Coding* 


1 

Sex  All 

1  Hale 

0 

2  Female 

1 

Ethnl  ri  tv  All 

i.  wnice 

1  fl 

nn 
uu 

3  Hispanic 

01 

4  Asian  American 

10 

^  Aiuei;  4. can  xnuxan 

JLU 

o  unc  xa^^s  i.L  xeu 

1  n 

JLU 

JLU 

STOC  All 

2  Low  Metro 

00 

3  High  Metro 

10 

Region  All 

1  Northeast 

000 

2  Southeast 

100 

010 

4  West 

001 

V/ V/  JL 

Parents'  Education  All 

1  <  High  School 

0000 

2  High  School  Grad 

1000 

3  Post-High  School 

0100 

4  College  Grad 

0010 

5  I  Don't  Know,  Missing 

0001 

G^'ade  All 

A 

00 

8 

11 

01 

Items  in  the  Home  All 

1  0  to  3  of  five  items 

00 

2  4  of  five  items 

10 

3  all  five  items 

01 

BLK  missing 

00 

*  Multicolumn  entries  without 

overbars  indicate  multiple 

contrasts. 

Barred  col\:imns  treated  as  one  contrast. 
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Tab^e  C-5  (continued) 


Contrast  Codings  for  1988  Writing  Trend  Conditioning  Variables 


Conditioning  Contrast 

Variable                           Ages                    Variable  Coding  Coding* 

Modal  age                              All                1  less  than  modal  age  10 

2  modal  age  CO 

3  greater  than  modal  age  01 

Homework                               All               1  Don't  have  any  00000 

2  0  done  10000 

3  less  than  1  hour  01000 

4  1  to  2  hours  00100 

5  2+  hours  00010 
BLK  missing  00001 

School  Type                          All               1  public  0 

2  private  1 

3  Catholic  1 


4  Bureau  of  Indian  Affairs 

5  Dept.  of  Defense 
missing 


TV  Watching  All  1  less  than  1  hour  0  00 

2  1  hour  1  01 

3  2  hours  2  04 

4  3  hours  3  09 

5  4  hours  4  16 

6  5  hours  5  25 

7  more  than  6  hours  6  36 
BLK  missing  3  09 

Mother's  wcirking  All  1  yes  1 

Outside  of  home  2  no  0 

BLK  missing  0 

Primary  language  All  1  En-^lish  00 

Spoken  in  the  home  2  Spanish  10 

3  other  01 

4  missing  01 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-5  (continued) 


Contrast  Codings  for  1983  Writing  Trend  Conditioning  Varlabl 

PQ 

Conditioning 

f!ont"'rA<:t" 

L 1  W  ^  Ci  O  W 

Variable  Ages 

Variable  Coding 

Codinp* 

Percent  Lunch  Program  All 

0 

000  0 

1 

001  0 

2 

002  0 

99 

099  0 

100 

100  0 

BLK  missine 

000  1 

Percent  of  White  All 

0-49  9    Minorttv  school 

00 

Students  in  School 

50-79  9  Integrated  school 

10 

80-100    Predomtnantlv  Whi 

01 

RLK  Missme 

01 

Grades  in  School  All 

1.  Mostly  A 

4.0 

2.  Mostly  B 

3.0 

3.  Mostly  C 

2.0 

4.  Mostly  D 

1.0 

5     L,e  <;  <;  th  an  D 

0.0 

6    between  A  and  B 

^/  *             ^>  ^  n  ^>  ^>  A  A      A  A      wA  A  A^A 

3.5 

7     bp twee n  R  and  C 

/     *                  ^  n            A  A                wA  A  A^A 

2.5 

8 .  between  C  and  D 

1.5 

9.  between  D  and  F 

0.5 

BLK  missing 

2.0 

Pages  a  Pay  Read  for  All 

1.  more  than  20 

100 

School  and  Homework 

2.  16  -  20 

100 

3.  11  -  15 

100 

4.  6-10 

010 

5.  less  than  5 

001 

BLK  missing 

000 

Number  of  Reports  All 

0  no  repoxtrs,  or  missing 

0 

Written 

1  one  report 

1 

2  two  reports 

2 

3  three  reports 

3 

4  four  reports 

4 

5  five  reports 

5 

6  six  reports 

6 

7  seven  reports 

7 

*  Multicolumn  entries  without 

overbars  indicate  multiple  contrasts. 

Barred  colximns  treated  as  one  contrast. 
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Table  C-6 

Contrast  Codings  for  1988  Civics  Cross-secticnal  Conditioning  Variables 


Conditioning 

Variable 

Variable 

Contrast 

Variable 

Grades 

Name 

Coding 

CodinE* 

Frequency  of 

4 

(Main  and 

S008701 

1  everyday 

1000 

Social  Studies 

Intercorr . 

) 

2  3-4/wk 

0100 

Class 

3  1-2/wk 

0010 

4  less  than  1/wk 

0001 

5  never  or  hardly 

ever  0000 

missing 

0000 

Studied  Govern- 

4 

(Main  and 

P800501 

1  yes 

10 

ment 

Intercorr. 

) 

2  no 

00 

missing 

01 

Number  of 

4 

(Main) 

ST-ALOT 

0 

0 

subjects 

1 

1 

studied  a  lot 

2 

2 

(out  of  6) 

3 

4 

4 

5 

5 

6 

6 

Studied  Laws 

4 

(Main  and 

P800601 

1  a  lot 

1 

Intercorr. 

) 

2  some 

1 

3  not  at  all 

0 

missing 

0 

Studied  Judges 

4 

(i-Iain) 

PoOO/Ol 

1  a  lot 

1 

1 

and  Courts 

2  some 

1 

3  not  at  all 

u 

missing 

A 

u 

Studied 

4 

(Main  and 

P800801 

1  a  lot 

1 

President 

Interco.r 

) 

2  some 

1 

and  Leaders 

3  not  at  all 

0 

missing 

0 

Studied 

4 

(Main) 

P800901 

1  a  lot 

1 

Elections 

2  some 

1 

and  Voting 

3  not  at  all 

0 

missing 

0 

*  Multicolumn  entries  without  overbars  ind.'.cate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-6  (continued) 
Contrast  Codings  for  1988  Civics  Cross-sectional  Conditioning  Variables 


Conditioning 
Variable 

Studied 
Community 


Studied  Rights 
and  Responsi- 
bilities 


Frequency  of 
Current  Events 


Studied  Civics 
in  Fifth  Grade 


Grades 

4  (Main  and 
Intercorr . ) 


4  (Main) 


4  (Main) 


8  (Main  and 
Intercorr. ) 


Variable 
Name 

P801001 


P801101 


P802101 


P800101 


Variable 
Coding 

1  a  lot 

2  some 

3  not  at  all 
missing 

1  a  lot 

2  some 

3  not  at  all 
missing 


1 
2 
3 
4 
5 


every  day 

3-4/wk 

1-2/month 

few/year 

never 


missing 

1  yes 

2  no 

3  don't  know 
missing 


Contrast 
Coding* 

1 
1 
0 
0 

1 
1 
0 
0 

1000 
0100 
0010 
0001 
0000 
0000 

1 

0 

c 

0 


Studied  Civics 
in  Sixth  Grade 


Studied  Civics 
in  Seventh  Grade 


Studied  Civi^;:i 
in  Eighth  Grade 


8  (Main  and 
Intercorr. ) 


8  (Main  and 
Intercorr. ) 


8  (Main  and 
Intercorr. ) 


P800102 


P800103 


P800104 


1  yes 

2  no 

3  don't  know 
missing 

1  yes 

2  no 

3  don't  know 
missing 

1  yes 

2  no 

3  don't  know 
missing 


1 
0 
0 
0 

1 

0 
0 
0 

1 

0 
0 
0 


*  MulticoluniH  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-6  (continued) 
Contrast  Codings  for  1988  Civics  Cross-sectional  Conditioning  Variables 


Conditioning 
Variable 

Studied  Civics 
Incomplete  Set 
of  Answers 


Grades 

8  (Main  and 
Intercorr . ) 


Number  of  8  (Main) 

Subjects  Studied 
a  lot  (out  of  10) 


Studied 
Constitution 


Studied  Lavs 


Studied  Courts 


8  (Main  and 
Intercorr . ) 


8  (Main  and 
Intercorr . ) 


Variable 
Name 

STCV-INC 


ST-ALOT 


Variable 
Coding 


8  (Main  and  P801201 
Intercorr. ) 


Studied  Congress    8  (Main  and  P8O1202 

Intercorr . ) 


P801203 


P801204 


0 
1 
2 
3 
A 
5 
6 
7 
8 
9 

10 

1  a  lot 

2  some 

3  not  at  all 
missing 

1  a  lot 

2  some 

3  not  at  all 
missing 

1  a  lot 

2  some 

3  not  at  all 
missing 

1  a  lot 

2  some 

3  not  at  all 
missing 


Contrast 
Coding* 

1 
0 


0 
1 
2 
3 
4 
5 
6 
7 
8 
9 

10 

2 
2 
1 
0 

2 
2 
1 
0 

2 
2 
1 
0 

2 
2 
1 
0 


*  Multicolurcn  entries  without  overbars  indicate  multiple  contrasts, 
Barred  columns  treated  as  one  contrast. 
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Table  C-6  (continued) 


Contrast  Codings  for  1988  Civics  Cross -sectional  Conditioning  Variables 


Conditioning 
Variable 

Studied 
President 


Studied 

Political 

Parties 


Studied  State 
and  Local 
Government 


Studied 
Democracy 


Studied  Other 
Governments 


Grades 

8  (Main  and 
Intercorr. ) 


8  (Main  and 
Intercorr. ) 


8  (Main  and 
Intercorr. ) 


8  (Main) 


Studied  Rights  &  8  (Main) 
Respons  ibil i  ties 


Active  in  4 
Areas  of  Civics 
(1.  2,  5,  8) 

Active  in  2 
Areas  of  Civics 
(3,  10) 


8  (Main) 


8  (Main) 


Variable 
Name 

P801205 


8  (Main  and  P801206 
Intercorr. ) 


P801207 


P801208 


P801209 


P8012iu 


ovA-ACT 


CVB-ACT 


Variable 
Coding 

1  a  lot 

2  some 

3  not  at  all 
missing 

1  a  lot 

2  some 

3  not  at  all 
missing 

1  a  lot 

2  some 

3  not  at  all 
missing 

1  a  lot 

2  some 

3  not  at  all 
siissing 

1  a  lot 

2  some 

3  not  at  all 
missing 

1  a  lot 

2  some 

3  not  at  all 
missing 

1 
0 


1 
0 


Contrast 
Coding* 

2 
2 
1 
0 

2 
2 
1 
0 

2 
2 
1 
0 

2 
2 
1 
0 

2 
2 
1 
0 

2 
2 
1 
0 

1 
0 


1 
0 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts, 
Barred  columns  treated  as  one  contrast. 
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Table  C-6  (continued) 


Contrast  Codings  for  1988  Civics  Cross -sectional  Conditioning  Variables 


Conditioning 
Variable 

Active  in  2 
Areas  of  Civics 
(4.  9) 

Active  in  2 
Areas  of  Civics 
(6.  7) 

Not  Active  in 
2  Areas  of 
Civics  (6,  7) 

Difficulty 
Reading  Text 


Grades 
8  (Hain) 

8  (Main) 

8  (Main) 

8  (Hain) 


Variable 

Name 

CVC-ACT 


CVD-ACT 


CVD-NOT 


S00880I 


Variable 
Codtnp 


Studied  Civics  12  (Main  and  Pbv0201 
in  Ninth  Grade  Intercorr.) 


Studied  Civics  12  (Main  an'i  P800202 
in  Tenth  Grade  Intercorr.) 


Studied  Civics  12  (Main  and  P800203 
in  Eleventh  Grade  Intercorr.) 


1 

0 


1 
0 


1 
0 


1  a  lot 

2  some 

3  none 
missing 

1  yes 

2  no 

3  don't  know 
0 

1  yes 

2  no 

3  don' c  know 
0 

1  yes 

2  no 

3  don't  know 
0 


Contrast 

Coding* 

1 
0 


1 
0 


1 
0 


i 
1 
0 
1 

1 
0 
0 
0 

1 
0 
0 
0 

1 
0 
0 
0 


Studied  Civics  12  (Main  and  P800204 
in  Twelfth  Grade  Intercorr.) 


1  yes 

2  no 

3  don't  know 
missing 


1 
0 
0 
0 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-6  (continued) 


Contrast  Codings  for  1988  Civics  Cross-sectional  Conditioning  Variables 


Conditioning                                      Variable               Variable  Contrast 

Variable                  Grades                 Name                    Coding  Coding* 

Studied  Civics       12  (Main  and         STCV-INC         \  1 

Incomplete  Set            Intercorr,)                           0  0 
of  Answers 

Years  of                 12  (Main  and         GOVT               0  0 

Government  Study         Intercorr,)                             ,25  ,25 

You  Have  Had                                                             ,5  ,5 

1-0  1,0 

2,0  2,0 

Advanced                 12  (Main  and         P800401           1  yes  1 

Placement                    Intercorr,)                           2  no  0 

missing  0 

Number  of               12  (Main  and         oT-ALOT          1  1 

Subjects  Studied         Intercorr,)                           0  0 
a  Lot  (out  cf  10) 

Average  Level         12  (Main  and         ST-AVE            1  1 

of  Study  for  10           Intercorr,)                           0  0 
Subjects 

Level  of                 12  (Main)              CIV- INT          3  3 

Interest  in                                                            2  2 

Civics                                                                    1  1 

1  1 

0  0 

Mock  Election         12  (Main)              P801501           1  several  times  1 

Trials                                                                    2  once  or  twice  1 

3  never  0 

missing  0 

Degree  of                12  (Main)              GOVATTN           2  2 

Attention                                                               1  1 

Government  Pays                                                   0  0 


*  Hulticolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-6  (continued) 


Contrast  Codings  for  IPhu  Civics  Cross- sectional  Conditioning  Variables 


Conditioning 
Variable 

Took  Civic« 
Class 


Grades 
12  (Main) 


Amount  of  Civics  12  (Main) 
Homework 


Civics  Grades 
in  School 


Active  in  4 
Are*"  ,  of  Civics 
(1.  2,  5.  8) 

Active  in  2 
Areas  of  Civics 
(3.  10) 

Active  in  2 
Areas  of  Civics 
(4,  9) 

Active  in  2 
Areas  of  Civics 
(6.  7) 


12  (Main) 


12  (Main) 


12  (Main) 


12  (Main) 


12  (Main) 


Variable 
Name 

HADCIV 
(P801801) 


CVHWORK 


Variable 
Coding 


CVGRADES 


CVA-ACT 


CVB-ACT 


CVC-ACT 


CVD-ACT 


1  yes 

2  no 
missing 


I 
2 
3 
4 
5 

4.0 
3,5 
3.0 
2.5 
2.0 
1.5 
1.0 
.5 

1 
0 


1 
0 


1 
0 


1 
0 


Contrast 
Coding* 

1 
0 
0 


1 
2 
3 
4 
5 

4.0 
3.5 
3.0 
2.5 
2.C 
1.5 
1.0 
.5 

1 
0 


1 
0 


1 
0 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-6  (continued) 
Contrast  Codings  for  1988  Civics  Cross -sectional  Conditioning  Variables 


Conditioning 
Variable 

Not  Very  Active 
in  These  2  Areas 
of  Civics 


Grades 
12  (Main) 


Variable 
Name 

CVD-NOT 


Variable 
Coding 


1 
0 


Contrast 
Coding* 

1 
0 


At  Least  Some 
Difficulty  in 
Reading  or  Missing 


12  (Main) 


RDDIF 


1 
0 


1 
0 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-7 


Contrast  Codings  for  Civics  Trend  Conditioning  Variables 


Conditioning 
Variable 

Gender 


Ethnicity 


Ages  Year 

13,  17  1976 

1982 
1988 

13,  17  197C 


1982 
1988 


STOC 


Region 


13,  17  1976 


Variable 
Name 

SEX 

SEX 
DSEX 

COLLRACE 


COLLRACE 
RACE 


13,  17      1976  STOC 


1982  STOC 
1988  STOC 


REGOBE 


1982  REGOBE 
1988  REGION 


Variable 
Coding 

1  Male 

2  Female 
see  1976-SEX 
see  1976-SEX 

1  White 

2  Black 

3  Hispanic 

4  Other 
BLK  missing 

see  197 6 -COLLRACE 

1  White 

2  Black 

3  Hispanic 

4  Asian  Am. 

5  Am.  Ind. 

6  Unclass. 
Blk,miss 

2  Low  Metro 
i  High  Metro 
1,4-7 ,miss 
see  1976-STOC 
see  1976-STOC 

1  Northeast 

2  Southeast 

3  Central 

4  West 

see  1976 -REGOBE 
see  1976 -REGOBE 


Contrast 
Coding* 

0 
1 


000 
100 
010 
001 
001 

000 
100 
010 
001 
001 
001 
001 

00 
10 
01 


000 
100 
010 
001 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-7  (continued) 


Contrast  Codings  for  Civics  Trend  Conditioning  Variables 


Conditioning 
Variable 

Parents'  Ed. 


Ages 
13,  17 


Variable 
Name 


Year 
1976  PARED 


Variable 
Coding 

1  <  hs 

2  hs  grad 

3  post-hs 
Blk,miss 


Contrast 
Coding* 

000 
100 
010 
001 


Parents'  Ed.  2       13,  17  1982 


PPARED 
(derived) 


1988 


PARED 


Items  in  the  Home  13,  17  1976 


1982 
1988 


HOMEENV 


HOMEEW 
H0MEEN2 


1  <  hs  0000 

2  hs  grad  1000 

3  post-hs  0100 

4  col  grad  0010 
Blk, miss, don't  know  0001 

1  <  hs  0000 

2  hs  grad  1000 

3  post-hs  0100 

4  col  grad  0010 
Blk, miss, don't  know  0001 

10-2  items  00 

2  3  items  10 

3  4  items  01 
Blk, miss, don't  know  00 
see  1976 -HOMEENV 

see  1976 -HOMEENV 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-7  (continued) 
Contrast  Codings  for  Civics  Trend  Conditioning  Variables 


Conditioning 
Variable 

TV  Watching 


Ages 


Year 


Home  Language 
Minority 


13 


17 


Variable 
Name 


Variable 
Coding 


Contrast 
Coding* 


1976 

not  avail. 

1982 

EO  7.5  301 

1  none 

0  00 

2  1  hr.less 

1  01 

3  1  hr 

1  01 

4  2  hrs 

2  04 

S  3  hr<5 

3  09 

6  4  hrs 

4  16 

7  5  hrs 

5  25 

8  6  or  more 

6  36 

Blk,miss 

3  09 

1988 

B001801 

1  none 

0  00 

2  1  hr,less 

1  01 

3  2  hrs 

2  04 

4  3  hrs 

3  09 

5  4  hrs 

4  16 

A   S  Virc 

Q    J  ilLo 

5  25 

7  6  or  more 

6  36 

Blk,miss 

3  09 

1976 

not  avail. 

1982 

E075602 

1  never 

0 

2  sometimes 

1 

3  often 

1 

Blk,miss 

0 

1988 

B003201 

1  never 

0 

2  sometimes 

1 

3  always 

1 

Elk, miss 

0 

1976 

OTHIANG 

1  often 

1 

2  sometimes 

1 

3  never 

0 

Blk,miss 

0 

1982 

E075602 

see  1976-OTHLANG 

0 

1988 

B003201 

1  never 

2  sometimes 

1 

3  always 

1 

Blk,miss 

0 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-  7  (continued) 


Contrast  Codings  for  Civics  Trend  Conditioning  Variables 


Conditioning 
Variable 

Homework 


Ages  Year 

13,  17  1976 
1982 


Variable 
Name 

not  avail. 
E075801 


1988 


B0G3901 


X  White  in  School  13,  17  1976 


PCTWHITE 


1982 


1988 


PCTWHITE 


PCTWHT 


Variable 
Coding 


1  don't  have 

2  don't  do 

3  <  1  hr 

4  1-2  hrs 

5  >  2  hrs 
Blk,miss 

1  don' t  have 

2  dcn't  do 

3  1/2  hr 

4  1  hr 

5  2  hrs 

6  >  2  hrs 
Bnc,miss 


1 
2 
3 
4 
5 
6 
7 


none 
0-9X 
10-19X 
20-29X 
30-39X 
40-49X 
50-59X 

8  60-69X 

9  70-79X 

10  80-89X 

11  90-99X 

12  lOOX 
Blk,miss 

0-49  white  minor. 
50-79  integ. 
80-100  white 
Blk,miss 

see  1982 -PCTWHITE 


Contrast 
Coding* 


100 
010 
Oil 
012 
013 
000 
100 
010 
Oil 
012 
012 
013 
000 

10 
10 
10 
10 
10 
10 
01 
01 
01 
00 
00 
00 
00 
10 
01 
00 
00 


*  Hulticolumn  entries  without"  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-7  (continued) 
Contrast  Codings  for  Civics  Trend  Conditioning  Variables 


Conditioning 
Variable 

Grade 


Ages 
13 


Year 


1976 


17 


Public  V. 
Private  School 


13,  17 


1982 
1988 


1976 
1982 


1988 


Variable 
Name 

COLLGRD 


1982  COLLGRD 
1988  MODGRD 


1976  COLLGRD 


COLLGRD 
MODGRD 


not  avail. 
PUBPRVSC 


SCHTYPE 


Variable 
Coding 

1  <  modal  gr 

2  7th  gr 

3  8th  gr 

4  >  modal  gr 
see  1976- COLLGRD 

1  <  modal  gr 

2  8th  gr 

3  >  modal  gr 

1  <  modal  gr 

2  10th  gr 

3  11th  gr 

4  >  modal  gr 
see  1976 -COLLGRD 

1  <  modal  gr 

2  11th  gr 

3  >  modal  gr 


1  public 

2  other 
Blk,miss 

1  public 

2  private 

3  Catholic 

4  B.  Ind. 

5  D.  Defense 


Blk.mlss 


Contrast 
Coding* 

00 
00 
10 
01 

00 
10 
01 

00 
00 
10 
01 

00 
10 
01 


0 

1 
1 

0 

1 
1 
1 
1 
1 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 


Table  C-7  (continued) 
Contrast  Codings  for  Civics  Trend  Conditioning  Variables 


Conditioning 

Variable  Ages 

Grades  in  School  13 


Year 


Variable 
Name 


Variable 
Coding 


17 


H.S.  Program 


17 


Contrast 
Coding^ 


1976 

not  avail. 

1982 

E075901 

1  A 

4.0 

2  A-B 

3.5 

3  B 

3.0 

4  B-C 

2.5 

5  C 

2.0 

6  C-D 

1.5 

7  D 

1.0 

8  <  D 

0.5 

Blk.miss 

2.0 

1988 

B005401 

see  1982-E075901 

1976 

GRADES 

1  A 

4.0 

2  A-B 

3.5 

3  B 

3.0 

4  B-C 

2.5 

5  C 

2.0 

6  C-D 

1.5 

7  D 

1.0 

8  <  D 

0.5 

Blk,miss 

2.0 

1982 

E075901 

see  1976-GRADES 

1988 

B005401 

see  1976-GRADES 

1976 

HSPROG 

1  general 

00 

2  col  prep 

10 

3  agriculture 

01 

4  business 

01 

5  distributive 

01 

6  health 

01 

7  home  ec 

01 

8  industrial 

01 

Blk,miss 

00 

1982 

E076001 

1  general 

00 

2  col  prep 

10 

3  vocational 

01 

Blk,miss 

00 

1988 

B005001 

see  1982-E076001 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-7  (continued) 


Contrast  Codings  for  Civics  Trend  Conditioning  Variables 


Variable 

Variable 

Contrast 

V«5i*ialilG 

Year 

Name 

Coding 

Coding* 

T 1*     '  lO  C     1  71 

17 

1976 

E141801 

1  true 

10 

\oU  o 

9  <»oinpwHat 

01 

3  not  true 

00 

Blk,miss 

01 

1982 

not  avail. 

1988 

P802501 

see  1976-E141801 

17 

1976 

E141802 

1  true 

10 

9  somewhat 

01 

3  not  true 

00 

Blk.miss 

01 

1982 

not  avail . 

1988 

P802601 

see  1976-E141802 

Nrt  f*Vi  4  T»  or  noTiT 

17 

1976 

E141803 

1  true 

10 

9  ^ompwHat 

01 

3  not  true 

00 

Blk,miss 

01 

1982 

no  t  ava  i 1 

1988 

P802701 

see  1976-E141803 

Knowledge  to 

17 

1976 

E141804 

1  true 

10 

Particlnate 

2  somewhat 

01 

3  no  t  true 

00 

Blk, miss 

01 

1982 

not  avail. 

1988 

P802801 

see  1976-E1418C4 

National 

17 

1976 

E141401 

1  every  day 

100 

Discussions 

2  1-2/wk. 

010 

3  3-4/month 

O.'l 

4  hardly  ever 

000 

Blk, miss 

010 

1982 

not  avail. 

1988 

P802901 

see  1976-E141401 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-7  (continued) 


Contrast-  Codings  for  Civics  Trend  Conditioning  Variables 


Conditioning 

Variable 

Variable 

Contrast 

Variable 

Ages 

Xeai: 

Name 

Coding 

Coding* 

Intetnational 

17 

1976 

E141402 

1  every  day 

100 

Discussions 

2  1-2/wk, 

010 

3  3-4/month 

001 

4  hardly  ever 

000 

Blk,miss 

010 

1982 

not  avail. 

1988 

P803001 

see  1976-E141402 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts • 
Barred  columns  treated  ^s  one  contrast • 
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Table  C-8 

Contrast  Codings  for  1988  U.S.  History  Cross-sectional  Conditioning  Variables 


Conditioning 
Variable 

How  often  have  social 
studies  class 


Age  Variable 
Classes  Name 

9  HIS  TAKE 


Ever  studied  history  9 
of  our  countiry 

Studied  early  explorers  9 


STUD. HIS 


STUD. EXP 


Variable 
Coding 


Contrast 
Coding* 


Never,  hardly  ever 

or  missing  0 

Less  than  once  a  week  1 

Once  or  twice  a  week  2 
Three  or  four  times 

a  week  3 

Svery  day  4 


No  or  missing 
Yes 


0 
1 


Not  at  all  or  missing  0 
Some  1 
A  lot  2 


Studied  Indians 


STUD.IND  Not  at  all  or  missing  0 

Some  1 
A  lot  2 


Studied  pilgrims 


STUD.PIL  Not  at  all  or  missing  0 

Some  1 
A  lot  2 


Studied  George 
Washington 


STUD.G^^  Not  at  all  or  missing  0 

Some  1 
A  lot  2 


Studied  pioneers 


STUD.PIO  Not  at  all  or  missing  0 

Some  1 
A  lot  2 


Studied  slavery 


STUD. SLA  Not  at  all  or  missing  0 

Come  1 
A  lot  2 


Studied  people  who 
invent  things  and 
make  new  discoveries 


STU1*).INV  Not  at  all  or  missing  0 

Some  1 
A  lot  2 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-8  (continued) 


Contrast  Codings  for  1988  U.S.  History  Cross -sectional  Conditioning  Vari^^Mes 


Conditioning  Age         Variable  Variable  Contrast 

VAriable  Classes         Name  Coding  Coding* 


Study  U.S.  history  in 

13 

HIS  TAKE 

Not  studied 

0 

grades  5,  6,  7,  8 

Studied  in  2  grades 

2 

Studied  in  3  grades 

3 

Studied  in  A  crad^^ 

4 

Studied  early  periods 

13 

EARLY  H 

Not  at  ail 

0 

of  U«S.  history 

JL 

A  lot 

0 

Studied  later  periods 

13 

POST  CH 

Not  at  all 

0 

of  U.S.  history 

Some 

1 

JL 

A  lot 

2 

Social  studies  teacher 

13 

TEACH  H 

Never 

0 

asks  you  to  do  things 

Few  times  a  year 

1 

Once  or  twice  a  month 

2 

3 

Almost  every  day 

k 

Difficulty  reading  social 

13 

TEXTBOOK 

None 

0 

studies  textbook 

Some 

1 

A  lot 

2 

Study  ^,S.  history  in 

17 

HIS  TAKE 

Not  studied 

0 

grade  9,  10,  11,  12 

Studied  in  1  grade 

1 

Studied  in  2  grades 

2 

Studied  in  3  .grades 

3 

Studied  in  4  grades 

4 

U.S.  history  coursework 

17 

HIS  COUR 

None 

00 

completed 

<  half  a  year 

25 

Half  a  year 

50 

Half  a  year  to  1  year 

75 

One  year 

100 

More  than  one  year 

200 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts « 
Barred  columns  treated  as  one  contrast. 
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Table  C-8  (continued) 
Contrast  Codings  for  1988  U.S.  History  Cross -sectional  Conditioning  Variables 


Conditioning 

Age 

Variable 

Variable  Contrast 

Variable 

Classes 

Name 

Codinc 

Coding* 

Studied  early  periods  of 

17 

AM  HIST 

Not  at  all 

0 

U.S.  history 

Some 

1 

A  lot 

2 

Studied  later  periods  of 

17 

WORLD  H 

Not  at  nil 

0 

U.S.  history 

Some 

1 

A  lot 

2 

Studied  minorities  and 

17 

MIN/WOM 

Not  at  all 

0 

women 

Some 

1 

A  lot 

2 

Like  studying  U.S. 

17 

LIKE  H 

Never  studied 

0 

history 

Like  other  subjects 

better 

1 

Interesting 

2 

One  of  favorites 

3 

Ever  had  U,S.  history 

17 

HAD  HIST 

No 

0 

Yes 

1 

Time  spent  each  week  on 

17 

HOMEWK  H 

None  assigned 

0 

U.S.  history  homework 

Haven' t  done  it 

0 

Less  than  1  hour 

1  hour 

1 

2  hours 

2 

3  hours 

6 

4  hours 

4 

5  hours  or  more 

S 

Grades  in  U.S.  history 

17 

GRADES  H 

Mostly  A 

aTo 

Half  A  and  helf  B 

3.5 

Mostly  B 

3.0 

Half  B  and  half  C 

2.5 

Mostly  C 

2.0 

Half  C  and  half  D 

1.5 

Mostly  D 

1.0 

Mostly  below  D 

0.0 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-8  (continued) 
Contrast  Codings  for  1988  U.S.  History  Cross -sectional  Conditioning  Variables 


Conditioning  Age         Variable  Variable  Contrast 

Varial^^fi  Classes         ^ame  Coding  Coding* 

Teacher  asks  you  to  17  TCACH/H  Never  0 

do  things  In  class  Few  times  a  year  1 

Once  or  twice  a  month  2 

Once  or  twice  a  week  3 

Almost  every  day  4 

Difficult  reading  17  TEXTBOOK  None  0 

U.S.  history  textbook  Some  1 

A  lot  2 


ERLC 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  coluans  treated  as  one  contrast. 


548 


Table  C-9 

Contrast  Codings  for  1988  Geography  Cross -sectional  Conditioning  Variables 


Conditioning 

Age 

Variable 

Variaole 

Contrast 

Variable 

Classes 

Neme 

Coding 

Codinc^ 

Take  geography  course 

17 

GEO  TAKE 

Not  take 

0 

in  grades  9,  10,  11,  12 

Take  in  1  grade 

1 

Take  in  2  grades 

2 

Take  in  3  grades 

3 

Take*,  in  4  grades 

k 

Studied  physical 

17 

PHYSICAL 

Not  at  all 

0 

eeography 

Very  little 

1 

Some 

2 

A  lot 

3 

Studied  social 

17 

SOCIAL 

Not  at  all 

0 

geography 

Very  little 

1 

Some 

2 

A  lot 

3 

World  history  and 

17 

W  HIS/G 

No,  haven't  taken 

0 

geography  coursework 

Yes 

1 

State/regional  geography 

17 

STAT/REG 

No,  haven't  taken 

0 

coursework 

Yes 

1 

U.S.  geography 

17 

US  GEO 

No,  haven't  taken 

0 

coursework 

Yes 

1 

U.S.  history  and 

17 

US  H/G 

No,  haven't  taken 

0 

geography  coursework 

Yesl 

Physical  geography 

17 

PHY/GEO 

No,  haven't  taken 

0 

coursework 

Yes 

1 

Economic  and  political 

17 

ECON/P 

No,  haven't  taken 

0 

geography  c^t^rsework 

Yesl 

Hunan  and  cultural 

17 

HW/CULT 

Nc,  haven't  taken 

0 

geography  coursework 

Yes 

1 

Urban  geography 

17 

URBAN  G 

No,  haven't  taken 

0 

coursework 

Yes 

1 

*  Hulti^  jlumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  tr-*  ted  as  one  contrast. 
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Table  C-10 


Contrast  Codings  for  1988  Mathematics  Trend  Conditioning  Variables 

Conditioning  Age  Contrast 

Variable  Classes  Variable  Coding  Coding* 

Overall  All  1 

Gender  All  1  Male  0 

2  Female  1 

Observed  Race  All  1  White  000 

2  Black  100 

3  Hispanic  010 

4  Asian  American  001 

5  American  Indian  000 

6  Other  000 
BLK  Missing  000 

STOC  All  1,4-7  all  except  2&3  00 

2  low  metro  10 

3  high  metro  01 

Region  All  1  Northeast  000 

2  Southeast  100 

3  Central  010 

4  West  001 

Parents'  Education  All  1  <  High  school  0000 

2  -  High  school  1000 

3  >  High  school  0100 

4  Graduated  College  0010 

5  Unknown,  missing  0001 

Modal  grade  All  1  <  modal  grade  10 

2  -  modal  grode/missing  00 

3  >  modal  grade  01 

Items  in  the  Home  All  1  0-2  items  00 

2  3  items  10 

3  4  items  01 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-10  (continued) 


Contrast  Codings  for  1988  Mathematics  Trend  Conditioning  Variables 


Conditioning 
Variable 


Age 
Classes 


Contrast 


Observed  Race  x  Gender  All 
("White"  includes 

American  Indian, 

Other,  and 

Unclassified. ) 


Observed  Race  x 
Parents '  Education 
("White"  includes 
American  Indian, 
Other,  and 
Unclassified) 


All 


School  Type 


All 


Variable  Coding 

Coding* 

1  White, Male 

000 

2  Black, Male 

000 

3  Hispanic, Male 

000 

4  Asian  American, Male 

000 

5  White, Female 

000 

6  Black, Female 

100 

7  Hispanic, Female 

010 

8  Asian  American, Female 

001 

1  White,  <HS 

0000 

0000 

0000 

2  White,  HS  Grad 

0000 

0000 

0000 

3  White,  HS  + 

0000 

0000 

0000 

4  White,  Col  Grad 

0000 

0000 

0000 

5  White,  missing 

0000 

0000 

0000 

6  Black,  <HS 

0000 

0000 

0000 

7  Black,  HS  Grad 

1000 

0000 

0000 

8  Black,  HS  + 

0100 

0000 

0000 

9  Black,  Col  Grad 

0010 

0000 

0000 

10  Black,  missing 

0001 

0000 

0000 

11  Hispanic,  <HS 

0000 

0000 

0000 

12  Hispanic,  HS  Grad 

0000 

1000 

0000 

13  Hispanic,  HS  + 

0000 

0100 

0000 

14  Hispanic,  Col  Grad 

0000 

0010 

uuuu 

15  Hispanic,  missing 

0000 

0001 

0000 

16  Asian  Amer. ,  <HS 

0000 

0000 

0000 

17  Asian  Amer. ,  HS  Grad 

0000 

0000 

1000 

18  Asian  Amer. ,  HS  + 

0000 

0000 

0100 

19  Asian  Amer.,  Col  Grd 

0000 

0000 

0010 

20  Asian  Amer.,  missing 

0000 

0000 

0001 

1  Public 

0 

2  Private 

1 

3  Catholic 

1 

4  Bureau  Indian  Affairs 

1 

5  Department  of  Defense 

1 

*  Mult'column  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-10  (continued) 
Contrast  Codings  for  1988  Mathematics  Trend  Conditioning  \7ariables 


Conditioning 
Variable 

TV  Watching 


Homework 


Age 
Classes 

All 


13.17 


Language  in  the  Home  9 


13,17 


Observed  Race  x  All 
Language  in  the  Home 


Variable  Coding 


Contrast 
Coding* 


1-3     0-2  hours 

100 

4-6    3-5  hours 

010 

7       6  +  hours 

001 

mi     i  nu 

Ul  ^  ^  ^  ^  1 

000 

100 

2  Didn't  do 

010 

3  1/2  hour  or  less 

012 

4  1  hour 

013 

S  2  VioiiT"*; 

6  MnTA  t*Vifln  2  Vioii"r<! 

000 

UI J  O  O  X  i  1^ 

000 

1  Never 

00 

2  Sometimes 

10 

3  Always 

01 

1  Never 

00 

2  Occasionally 

01 

3  About  half  the  time 

01 

4  Most  of  the  time 

10 

5  Always 

10 

1  White,  Of  ten 

GO  00 

00 

2  White, Sometimes 

00  00 

00 

3  White, Never 

00  00 

00 

4  Black, Often 

10  00 

00 

5  Black, Sometimes 

01  00 

00 

6  Black, Never 

00  00 

00 

7  Hispanic, Of ten 

00  10 

00 

8  Hispanic, Some 

00  01 

00 

9  Hispanic, Never 

00  00 

00 

10  Asian  Am. , Often 

00  00 

10 

11  Asian  Am. ,Some 

00  00 

01 

12  Asian  Am., Never 

00  00 

00 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  coluipns  treated  as  one  contrast. 
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Table  C-10  (continued) 


Contrast  Codings  for  1988  Mathematics  Trend  Conditioning  Variables 

Conditioning  Age  Contrast 

Variable  Classes  Variable  Coding  Coding* 

NMATH  17  1  Pre -Algebra  10000 

2  Algebra  01000 

3  Geometry  00100 

4  Algebra  2  00010 

5  Calculus  00001 

6  Something  Else  00000 

Computer  Prog*  17  1  Taken  1 

Class  2  Not  Taken  0 

Grades  13,17       1  A  aTo 

2  A-B  3.5 

3  B  3.0 

4  B-C  2.5 

5  C  2.0 

6  C-D  1.5 

7  D  1.0 

8  <  D  0.5 
Missing  2.0 

Type  of  Mi»th  Class  13  1  None  000 

2  Regular  Math  100 

3  Pre- Algebra  010 

4  Algebra  001 

5  Other  001 
Missing  000 

r.rc  You  Studying  9,13         1  Ves  1 

Computers  2  No  0 

High  School  Program         17  1  General  00 

2  College  Preparatory  10 

3  Vocational, Technical  01 
missLng  00 


*  Multicoiumn  entries  v;ithout  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-10  (continued) 


Contrast  Codings  for  19^3  Mrthematics  Trend  Conditioning  Variables 


Conditioning                    Age  Contrast 

Variable                    Classes            Variable  Coding  Coding* 

Derived  Race                   All          1  White  000 

2  Black  100 

3  Hispanic  010 

4  Asian  American  001 

5  Other  000 
missing  000 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Table  C-11 


Contrast  Codings  for  1988  Science  Trend  Conditioning  Variables 

Conditioning  Age  Contrast 

Variable,  Classes  Variable  Coding  Coding* 

Overall  All  1 

Gender  All  1  Male  0 

2  Female  1 

Observed  Race  All  1  White  000 

2  Black  100 

3  Hispanic  010 

4  Asian  American  001 

5  American  Indian  000 

6  Other  000 
BLK  Missing  000 

STOC  All  1,4-7  all  except  26c3  00 

2  low  metro  10 

3  high  metro  01 

Region  All  1  Northeast  000 

2  Southeast  100 

3  Central  010 

4  West  001 

Parents'  Education  All  1  <  High  school  0000 

2  -  High  school  1000 

3  >  High  school  0100 

4  Graduated  College  0010 

5  Unknown,  missing  0001 

hodal  grade  All  i.  <  modal  grade  10 

2  -  modal  grade/missing  00 

3  >  modal  grade  01 

Items  in  the  Home  All  10-2  items  00 

2  3  items  10 

3  4  items  01 

School  Type  All  1  Public  0 

2  Private  1 

3  Catholic  1 

4  Bureau  Indian  Affairs  1 

5  Department  of  Defense  1 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts. 
Barred  columns  treated  as  one  contrast. 
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Tables  C-12  through  C-43 
ESTIMATED  EFFECTS  FOR  1988  NAEP  CONDITIONING  VARIABLES 


RIC 
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Table  C-12 


Estimated  Effects  for  Reading  Cross-sectional  Conditioning  Variables 
1988  Focused-BIB  Sample,  Grade  4/Age  9 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-3. 

493459 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

0. 

088505 

SEX  (FEMALE) 

3 

ETHNIC2 

-0. 

293405 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC 3 

-0. 

316179 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

0. 

010243 

OBSERVED  ETHNICITY  (ASIAN) 

G 

ST0C2 

0. 

293781 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0. 

096133 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

0. 

075012 

REGION  (SOUTHEAST) 

9 

REGI0N3 

0. 

009300 

REGION  (CENTRAL) 

10 

REGI0N4 

0. 

027525 

REGION  (WEST) 

11 

PARED2 

0. 

344202 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0. 

509880 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0. 

438445 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

0. 

314739 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

H0MEITM3 

0. 

111460 

ARTICLES  IN  HOME  (YES  TO  3) 

16 

H0MEITM4 

0. 

161661 

ARTICLES  IN  HOME  (YES  TO  4) 

17 

TV.  1 

0. 

209353 

TV  WATCHING 

18 

TV.  2 

-0. 

034422 

TV  WATCHING 

19 

HOMELNGl 

-0 

020347 

OTHER  LANGUAGE  AT  HOME  (SOMETIMES,  ALWAYS) 

20 

HW-NO 

0 

220639 

HOMEWORK  (NONE  ASSIGNED) 

21 

HW-YES 

0 

222398 

HOMEWORK  (YES  -  SOME  AMOUNT) 

22 

HW-2345 

0 

006786 

HOMEWORK  AMOUNT  (LINEAR) 

23 

XLUNCH  1 

-0 

001664 

PERCENT  IN  LUNCH  PROGRAM 

24 

XLTJNCH  2 

-0 

.123354 

MISSING  LUNCH  PROGRAM 

25 

XWHITE  1 

-0 

.054569 

WHITE  MINORITY 

26 

XWHITE  2 

-0 

.031799 

INTEGRATED 

27 

AGE/GRDl 

-0 

. 604877 

MODAL  AGE,  <  MODAL  GRADE 

28 

AGE/GRD2 

0 

.010017 

MODAL  AGE,  MODAL  GRADE,  MISSING 

29 

AGE/GRD3 

0 

. 544105 

MODAL  AGE,  >  MODAL  GRADE 

30 

AGE/GRD4 

-0 

.193332 

>  MODAL  AGE,  MODAL  GRADE 

31 

SCH.TYPE 

0 

.182076 

PUBLIC  V.  PRIVATE  SCHOOLS 

32 

H/HOMEWK 

-0 

. 106614 

HELP  WITH  HOMEWORK  (EVERY  DAY,  TWICE  A  WEEK) 

33 

PRESCH 

0 

.173482 

WENT  TO  PRESCHOOL 

34 

PAR/HOME 

0 

.018582 

SINGLE/MULTIPLE  PARENT  (MULTIPLE) 

35 

MOTH/K'.M 

0 

.268381 

MOTHER  AT  HOME  (YES) 

36 

MOTH/WRK 

-0 

.065688 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

37 

GRUN  HOM 

0 

.098832 

GROWNUP  AT  HOME  RIGHT  AFTER  SCHOOL 

38 

PAGE/RDl 

0 

.129134 

PAGES  A  DAY  READ     (6  AND  UP) 

39 

PAGE/RD2 

-0 

.000352 

PAGES  A  DAY  READ     (11  AND  UP) 

40 

RDGCOMPl 

0 

.095330 

READING  COMPOSITE  #  1 

41 

RDGC0MP2 

0 

.060951 

READING  COMPOSITE  //  1 

(continued) 
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Table  C-12  (continued) 


Estimated  Effects  for  Reading  Cross-sectional  Conditioning  Variables 
1988  Focused-BIB  Sample,  Grade  4/Age  9 


Estimated 


Variable 

Effect 

Description 

42 

RDGBKGDl 

0 

.110859 

READING 

BACKGROUND  //  1 

43 

RDGBKGD2 

0 

.005445 

READING 

BACKGROUND  //  2 

44 

RDGBKGD3 

0 

.016781 

READING 

BACKGROUND  //  3 

45 

RDGBKGD4 

0 

.097556 

READING 

BACKGROUND  //  4 

46 

RDGBKGD5 

-0 

.072348 

READING 

BACKGROUND  //  5 

47 

RDGBKGD6 

0 

.009887 

READING 

BACKGROUND  //  6 

48 

RDGBKGD7 

-0 

.033549 

READING 

BACKGROUND  //  7 

49 

RDGBKGD8 

0 

.065660 

READING 

BACKGROUND  #  8 

50 

RDGBKGD9 

0 

.27?.768 

READING 

BACKGROUND  #  9 

51 

RDGBKGIO 

0 

.026865 

READING 

BACKGROUND  #  10 
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Table  C-13 


Estimated  Effects  for  Reading  Cross-sectional  Conditioning  Variables 
1988  Iptercorrelation  Sample,  Grade  4/Agc  9 


Estisiated 


Variable 

Effect 

Description 

1 

OVERALL 

-1.703461 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

0.296941 

SEX  (FEMALE) 

3 

ETHNIC2 

-0.349940 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNi:C3 

-0.361704 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

0.232422 

OBSERVED  ETHNICITY  (ASIAN) 

6 

ST0C2 

0.519909 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0.168985 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

0.176519 

REGION  (SOUTHEAST) 

9 

REGI0N3 

0.096705 

REGION  (CENTRAL) 

10 

REGI0N4 

0.053068 

REGION  (WEST) 

11 

PARED2 

0.108159 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0.440773 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0,407230 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

0.081511 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

H0MEITM3 

0.090329 

ARTICLES  IN  HOME  (YES  TO  3) 

16 

H0MEITM4 

0.164800 

ARTICLES  IN  HOME  (YES  TO  4) 

17 

TV,  1 

0.30G408 

TV  WATCHING 

18 

TV.  2 

-0.045694 

TV  WATCHING 

19 

HOMELNGl 

-0.006770 

OTHER  LANGUAGE  AT  *  ME  (SOMETIMES,  ALWAxS) 

20 

HW-NO 

-0.17S979 

HOMEWORK  (NONE  ASSIGNED) 

21 

HW-YES 

-0,187228 

HOMEWORK  (YES  -  SOME  AMOUNT) 

22 

Hw-2345 

-0.004841 

HOMEWORK  AMOUNT  (LINEAR) 

23 

X LUNCH  1 

-0.0036/2 

PERCENT  IN  LUNCH  PROGRAM 

24 

ZLUNCH  2 

-0.112552 

MISSING  LUNCH  PROGRAM 

25 

ZWHITE  1 

-0,096829 

WHITE  MINORITY 

26 

XWHITE  2 

-0.104274 

INTEGRATED 

27 

AGE/GRDl 

-0.636146 

MODAL  AGE,  <  MODAL  bRADh 

28 

AGE/GRD2 

0.015431 

MODAL  AGE,  MODAL  GRADE,  MISSING 

29 

AGE/GRD3 

0.190052 

MODAL  AGE,  >  MODAL  GRADE 

30 

AGE/GRD4 

-0.188559 

>  MODAL  AGE,  MODAL  GRADE 

31 

SCH.TYPE 

0.041692 

PUBLIC  V.  PRIVATE  SCHOOLS 

32 

H/HOMEWK 

-0.260516 

HELP  WITH  HOMEWORK  (EVERY  DAY,  TWICE  A  WEE<) 

33 

PRESCH 

0.20^281 

WENT  TO  PRESCHOOL 

34 

PAR/HOME 

0.040124 

SINGLE/MULTIPLE  PARENT  (MULTIPLE) 

35 

MOTH/HOM 

0.303563 

MOTHER  AT  HOME  (YES) 

36 

MOTH/WRK 

0.028529 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

37 

GRWN  HOM 

0.088326 

GROWNUP  AT  1"  ^.E  RIGHT  AFTER  SCHOOL 

38 

PAG£:/RD1 

0.119129 

PAGES  A  DAY  u£i\D    (6  AND  UP) 

39 

PAGE/RD2 

0.010113 

PAGES  A  DAY  READ    (11  AND  UP) 
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Table  C-14 


Estimated  Effects  for  Reading  Cross-sectional  Conditioning  Variable^; 
1988  Focused-BIB  Sample,  Grade  8/Age  13 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-2. 

878516 

OVERALL  CONSTANT  '1*  FOR  EVERYONE 

2 

GEND£R2 

0. 

095517 

SEX  (FEMALE) 

3 

ErriNIC2 

-0. 

200724 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC3 

-0. 

205606 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

0. 

130494 

OBSERVED  ETHNICITY  (ASIAN) 

6 

ST0C2 

0. 

072850 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

-0. 

016063 

SIZE  AND  TiZE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

-0. 

020760 

REGION  (SOUTHEAST) 

9 

REGI0N3 

-0. 

017065 

REGION  (CENTRAL) 

10 

REGI0N4 

-0. 

031544 

REGION  (WEST) 

11 

FARED2 

0. 

055548 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0.208053 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0. 

191380 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

FARED 

-0, 

024154 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

H0MEITM3 

0 

045925 

ARTICLES  IN  HOME  (YES  TO  3) 

16 

H0MEITM4 

0. 

073831 

ARTICLES  IN  HOME  (YES  TO  4) 

17 

TV.  1 

0. 

039022 

TV  WATCHING 

18 

TV.  2 

-0. 

006148 

TV  WATCHING 

19 

HOMELNGl 

-0. 

024874 

OTHER  LANGUAGE  AT  HOME  (SOMETIMES,  ALWAYS) 

20 

HW-NO 

0. 

423507 

HOMEV/ORK  (NONE  ASSIGNED) 

21 

HW-YES 

0 

519192 

HOMEWORK  (YES  -  SOME  AMOUNT) 

22 

HW-2345 

-0. 

017066 

HOMEWORK  AMOUNT  (LINEAR) 

23 

XLUNCH  1 

-0 

000706 

PERCENT  IN  LUNCH  PROGRAM 

24 

ZLUNCH  2 

-0 

088738 

MISSING  LUNCH  PROGRAM 

25 

XWHITE  1 

-0 

047834 

WHITE  MINORITY 

26 

XWHITE  2 

-0 

090714 

IfJTEGBATED 

27 

AGE/GRDl 

-0 

494470 

MODAL  AGE,  <  MODAL  GRADE 

28 

AGE/GRD2 

-0 

170739 

MODAL  AGE,  MODAL  GRADE,  MISSING 

29 

AGE/GRD3 

0 

067865 

MODAL  AGE,  >  MODAL  GRADE 

30 

AGE/GRD4 

-0. 

290966 

>  MODAL  AGE,  MODAL  GRADE 

31 

SCH.TYPE 

0 

179425 

PUBLIC  V.  PRIVATE  SCHOOLS 

32 

H/HOMEWK 

-0 

109597 

HFLP  WITH  HOMEWORK  (EVERY  DAY,  TWICE  A  WEEK) 

33 

PAR/HOME 

0 

005510 

SINGLE/MULTIPLE  PARENT  (MULTIPLE) 

34 

MOTH/HOM 

0. 

134401 

MOTHER  AT  HOME  (YES) 

35 

MOTH/WRK 

-0 

054176 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

36 

PAGE/RDl 

0 

126464 

PAGES  A  DAY  READ    (6  AND  UP) 

37 

PAGE/RD2 

0 

027989 

PAGES  A  DAY  READ    (11  AJTD  UP) 

38 

GRADRATE 

0 

269446 

DO  YOU  EXPECT  TO  GRADUATE  FROM  HIGH  SCHOOL 

39 

DAYSMISS 

0. 

010742 

DAYS  OF  SCHOOL  MISSEL  LAST  MONTH 

40 

GRADES 

0 

243183 

GRADES  IN  SCHOOL 

41 

RDGCOMPl 

0 

016368 

READING  COMPOSITE  //  1 

(continued) 
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Table  C-14  (continued) 


Estimated  Effects  for  Reading  Cross-sectional  Conditioning  Variables 
1988  Focused-BIB  Sample,  Grade  8/Age  13 


Estimated 


Variable 

Effect 

Description 

A  O 

RDGCUrlrZ 

-0 . 

AOT  QOO 

READING 

COMPOSITE  #  2 

A  O 

RDGCOMPi 

AO  ^  ^  O 1 

026681 

READING 

COMPOSITE  #  3 

44 

RDGC0MP4 

0 . 

A^  OT  eft 

062159 

T>  r*  A  TNT 

READING 

COMPOSITE  a  4 

45 

RDGBKGDl 

0 . 

AO  O  C  T  O 

03251Z 

READING 

D  A  r*  1//^D  AT  T^7T^ 

41 
It 

1 

46 

RDGBKGD2 

0 

AT  "7  O  "7  A 

017270 

READING 

UAPVPOOTTXin 
DiiONuKU  U  U 

Ji 
U 

z 

A  "7 

HI 

RDGBKGD3 

-0 

AAO  "7  Tft 

002779 

READING 

BACKGROUND 

a 

3 

A  O 
HO 

RDG6KGD4 

-0 

A  A  1  O  A  "7 
0^134/ 

n      A  TN  T 

READING 

BACKGROUND 

4 

49 

RDGBKGD5 

-0 

A  ^  O  Q  "5  O 

062y JO 

READING 

BACKGROUND 

# 

5 

RDGBKGD6 

A 
U 

AT  O Q  "7 A 
OlJi/4 

READING 

BACKGROUND 

6 

KUUdKOU / 

A 

-  u 

UIZ lUo 

BACKGROUND 

# 

7 

52 

RDGBKGOe 

0 

036475 

READING 

BACKGROUND 

8 

53 

RDGBKGD9 

0 

067985 

FADING 

BACKGROUND 

9 

54 

RDGBKGIO 

0 

005173 

READING 

BACKGROUND 

# 

10 

55 

RDGBKGll 

0 

.271575 

READING 

BACKGROUND 

11 

56 

RDGBKG12 

0 

.172633 

READING 

BACKGROUND 

12 

57 

RDGBKG13 

-0 

.037839 

READING 

BACKGROUND 

13 

58 

RDGBKG14 

-0 

.006027 

READING 

BACKGROUND 

14 

59 

RDGBKG15 

0 

.017113 

READING 

BACKGROUND 

15 

60 

RDGBKG16 

0 

.049804 

READING 

BACKGROUND 

16 

61 

RDGBKG17 

-0 

.007475 

READING 

BACKGROUND 

17 

62 

RDGBKG18 

-0 

.014529 

READING 

BACKGROUND 

18 

63 

RDGBKG19 

0 

.028883 

READING 

BACKGROUND 

19 

64 

RDGBKG20 

0 

.101803 

READING 

BACKGROUND 

20 
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Table  C-15 


Estimated  Effoct5>  for  Reading  Cross-sectional  Conditioning  Variables 
1988  Intercorrelation  Sample,  Grade  8/Age  13 


Estimated 


Variable 

Effect 

1 

OVERALL 

-2 

.030369 

2 

GENDER2 

0 

.166260 

3 

ETHNIC2 

-0 

.198242 

A 

ETHNIC 3 

-0 

.195727 

5 

ETHNIC4 

-0 

.066718 

6 

ST0C2 

0 

.159133 

7 

ST0C3 

0 

.131332 

8 

REGI0N2 

0 

.004577 

9 

REGI0N3 

0 

.070446 

10 

REGI0N4 

-0 

.012584 

11 

PARED2 

-0 

.020636 

12 

PARED3 

0 

.080334 

13 

PARED4 

0 

.059461 

14 

PARED_ 

-0 

.214610 

15 

H0MEITM3 

0 

.059626 

16 

H0MEITM4 

0 

.158479 

17 

TV.  1 

0 

.027980 

18 

TV.  2 

-0 

.010843 

19 

KOMELNGl 

0 

.011643 

20 

HW-NO 

0 

199523 

21 

HW-YES 

0 

443435 

22 

HW-2345 

0 

003335 

23 

XLUNCH  1 

-0 

000250 

24 

XLUNCK  2 

0. 

0?.4631 

25 

XWHITE  1 

0. 

018448 

26 

XWHITE  2 

-0. 

056145 

27 

AGE/GRDl 

-0. 

155339 

28 

AGE/GRD2 

0. 

140843 

29 

AGE/GRD3 

0. 

085695 

30 

AGE/GRD4 

0. 

016198 

31 

SCH.TYPE 

0. 

219716 

32 

H/HOMEWK 

-0. 

110421 

33 

PAR/KOME 

-0. 

073100 

34 

MOTH/HOM 

0. 

123323 

35 

MOTH/WRK 

-0. 

046198 

36 

PAGE/RDl 

0. 

101266 

37 

PAGE/RD2 

-0. 

020843 

38 

GRADRATE 

0. 

351427 

39 

DAYSMISS 

0. 

054705 

40 

GRADES 

0. 

330826 

Description 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 
SEX  (FEMALE) 

OBSERVED  ETHNICITY  (BLACK) 

OBSERVED  ETHNICITY  (HISPANIC) 

OBSERVED  ETHNICITY  (ASIAN) 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

REGION  (SOUTHEAST) 

REGION  (CENTRAL) 

REGION  (WEST) 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 
PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 
PARENTS  EDUCATION  (COLLEGE  GRAD) 
PARENTS  EDUCATION  (MISSING,  I  DON' „  KNOW) 
ARTICLES  IN  HOME  (YES  TO  3) 
ARTICLES  IN  HOME  (YES  TO  4) 
TV  WATCHING 
TV  WATCHING 

OTHER  LANGUAGE  AT  HOME  (SOMETIMES,  ALWAYS) 

HOMEWORK  (NONE  ASSIGNED) 

HOMEWORK  (YES  -  SOME  AMOUNT) 

HOMEWORK  AMOUNT  (LINEAR) 

PERCENT  IN  LUNCH  PROGRAM 

MISSING  LUNCH  PROGRAM 

WHITE  MINORITY 

INTEGRATED 

MODAL  AGE,  <  MODAL  GRADE 
MODAL  AGE,  MODAL  GRADE,  MISSING 
MODAL  AGE,  >  MODAL  GRADE 
>  MODAL  AGE,  MODAL  GRADE 
PUBLIC  V.  PRIVATE  SCHOOLS 

HELP  WITH  HOMEWORK  (EVERY  DAY,  TWICE  A  WEEK) 

SINGLE/MULTIPLE  PARENT  (MULTIPLE) 

MOTHER  A'^  HOME  (YES) 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

PAGES  A  DAY  READ    (6  AND  UP) 

PAGES  A  DAY  READ    (11  AND  UP) 

DO  YOU  EXPECT  TO  GRADUATE  FROM  HIGH  SCHOOL 

DAYS  OF  SCHOOL  MISSED  LAST  MONTH 

GRADES  IN  SCHOOL 


564 


6^3 


Table  C-16 


Estimated  Effecto  for  Reading  Cross -sectional  Conv'itlonlng  Variables 
1985  Focused-BIB  Sample,  Grade  12/Age  17 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-1. 

588959 

OVERALL  COiJSTAbiT  '1'  FOR  EVERYONE 

2 

GEHDER2 

0. 

062483 

SEX  (FEMALE) 

3 

ETHNIC2 

-0. 

265308 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC  3 

-0. 

163774 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

-0. 

043188 

OBSERVED  ETHNICITY  (ASI/-0 

6 

ST0C2 

0. 

125134 

SIZE  AND  TYP£  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0. 

090530 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

0. 

074697 

REGION  (SOUTHEAST) 

9 

REG ION 3 

0. 

067739 

REGION  (CENTRAL) 

10 

REGI0N4 

0. 

090882 

REGION  (WEST) 

11 

PARED2 

-0. 

057941 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0. 

018846 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0. 

005209 

Parents  education  (college  ghad) 

14 

PARED_ 

-0. 

133514 

fARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

H0MEITM3 

0 

119471 

ARTICLES  IN  HOME  (YES  TO  3) 

16 

H0MEITM4 

0 

109873 

ARTICLES  IN  HOME  (YES  TO  4) 

17 

TV.  1 

0 

069582 

TV  WATCHING 

18 

TV.  2 

-0 

013791 

TV  WATCHING 

19 

HOHELNGl 

-0 

063237 

OTHER  LANGUAGE  AT  HOME  (SOME  IMES,  ALWAYS) 

20 

HW-NO 

-0 

180651 

HOMEWORK  (NONE  ASSIjNED) 

21 

-0 

035056 

HOMEWORK  (YbS  -  SOME  ArlOUNi; 

22 

Hw-2345 

-0 

010146 

HOMEWORK  AMOUNT  (LINEAR) 

23 

X LUNCH  1 

0 

002396 

PERCENT  IN  LUNCH  PROGRAM 

24 

XLUNCH  2 

-0 

.018666 

MISSING  LUNCH  I'ROGRAM 

25 

XWHITE  1 

0 

'•01429 

WHITE  MINORITY 

26 

XWHITE  2 

0 

.014251 

INTEGRATED 

27 

AGE/GRDl 

-0.329255 

MODAL  AGE,  <  MODAL  GRADE 

28 

AGE/GRD2 

-0 

.207815 

MODAL  AGE,  MODAL  GRADE,  MISSING 

29 

AGE/GEU)4 

-0 

.337198 

>  MODAL  AGE,  MODAL  GRADE 

30 

SCH.TYPE 

0 

.019471 

PUBLIC  V.  PRIVATE  SCHOOLS 

31 

H/HOHEWK 

-0 

.180211 

HELP  WITH  HOMEWORK  (EVERY  DAY,  TWICE  A  W-EK) 

32 

PAR/H0f:2 

-0 

.018235 

SINGLE/MULTIPLE  PARENT  (MULTIPLE) 

33 

MOTH/HOM 

0 

.035154 

MOTHER  AT  HOME  (YES) 

34 

MOTH/WRK 

0 

.000284 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

35 

PAGE/RDl 

0 

.0826.86 

PAGES  A  DAY  READ     (6  AND  UP) 

36 

PAGE/RD2 

0 

.019541 

PAGES  A  DAY  READ     (11  AND  UP) 

37 

DAYSHISS 

0 

.041734 

DAYS  OF  SCHOOL  MISSED  LAST  MO.  iH 

38 

GRADES 

0 

.233978 

GRADES  IN  SCHOOL 

39 

HS  PGM2 

0.090736 

HIGH  SCHOOL  PROGRAM  (COLLEGE  PREPARATORY) 

'iO 

HS  PGM3 

-0 

.094364 

HIGH  SCHOOL  PROGRAM  (VOCATIONAL,  TECHNICAL) 

41 

POSTS EC2 

-0 

.00797^ 

POST- SECONDARY  PLANS  (TWO-YEAR  COLLEGE) 

(continued) 


ERIC 
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Table  C-16  (continued) 


Estimated  Effects  for  Reading  Cross-sectional  Conditioning  Variables 
1988  Focused-BIB  Sample,  Grade  12/Age  17 


Estimated 

Variable  Effect  Description 


42 

P0STSEC3 

0 

136958 

POST -SECONDARY  PLANS 

(FOUR-YEAR  COLLEGE) 

43 

WORKHOUR 

-0 

001018 

HOURS  OF  OUTSIDE  WORK 

44 

ENG. 23 

0 

053846 

TYPE  OF 

ENGLISH  CLASS (ADV.  PLACEMENT&COLLEGE) 

45 

ENGLISH5 

-0 

?24120 

TYPE  OF 

ENGLISH  CL^SS (REMEDIAL) 

46 

RDGCOMPl 

-0 

001806 

READING 

COMPOSITE  i 

f  1 

47 

RDGC0MP2 

-0 

017579 

READING 

COMPOSITE  i 

f  2 

48 

RDGC0MP3 

-0 

039289 

READING 

COMPOSITE  i 

}  3 

49 

RDGCOMPA 

0 

071240 

READING 

COMPOSITE  i 

f  4 

50 

RDGC0MP5 

0 

001928 

READING 

COMPOSITE  //  5 

51 

RDGC0MP6 

-0. 

047195 

READING 

COMPOSITE  #  6 

52 

RDGC0MP7 

-0 

032501 

READING 

COMPOSITE  //  7 

53 

RDGBKGDl 

0 

007427 

READING 

BACKGROUND 

// 

1 

54 

RDGBKGD2 

0 

020779 

READING 

BACKGROUND 

// 

2 

55 

RDGBKGD3 

-0 

002589 

READING 

BACKGROUND 

// 

3 

56 

RDGBKGD4 

-0 

031797 

READING 

BACKGROUND 

// 

4 

57 

RDGBKGD5 

-0 

021215 

READING 

BACKGROUND 

// 

5 

58 

RDGBKGD6 

-0.006863 

READING 

BACKGROUND 

// 

6 

59 

RDGBKGD7 

-0 

002056 

READING 

BACKGROUND 

// 

7 

60 

RDGBKGD8 

0 

002351 

READING 

BACKGROUND 

// 

8 

61 

RDGBKGD9 

0 

088956 

READING 

BACKGROUND 

// 

9 

62 

RDGBKGIO 

0 

043146 

READING 

BACKGROUND 

// 

10 

63 

RDGBKGll 

-0 

064605 

READING 

BACKGROUND 

// 

11 

64 

RDGBKG12 

-0 

040126 

READING 

BACKGROUND 

// 

12 

65 

RDGBKG13 

0 

003461 

READING 

BACKGROUND 

// 

13 

66 

RDGBKG14 

0 

195865 

READING 

BACKGROUND 

// 

14 

67 

RDGBKG15 

0 

142466 

READING 

BACKGROUND 

■It 

15 

68 

RDGBKG16 

-0 

006831 

READING 

BACKGROUND 

if 

16 

69 

RDGBKG17 

0 

031799 

READING 

BACKGROUND 

17 

70 

RDGBKG18 

-0 

050291 

READING 

BACKGRO"ND 

18 

71 

RDGBKG19 

-0 

022255 

READING 

BACKGROUND 

19 

72 

RDGBKG20 

0 

058058 

READING 

BACKGROW"" 

if 

20 

73 

RDGBKGll 

0 

011322 

READING 

BACKGROUb.1) 

if 

21 

74 

RDGBKG12 

-0 

027553 

READING 

BACKGROUND 

if 

22 

75 

RDGBKG13 

0 

084234 

RFJiDING 

BACKGROUND 

if 

23 

76 

RDGBKG14 

-0 

024406 

READING 

BACKGROUND 

if 

24 

77 

RDGBKG15 

0 

009928 

READING 

BACKGROUND 

if 

25 

78 

RDGBKG16 

0 

.008416 

READING 

BACKGROUND 

if 

26 

79 

RDGBKG17 

-0 

.050294 

READING 

BACKGROUND 

if 

27 

80 

RDGBKG18 

0 

.046429 

READING 

BACKGROUND 

if 

2P 

81 

RDGBKG19 

0 

.000222 

READING 

BACKGROUND 

if 

29 

82 

RDGBKG20 

0 

.005741 

READING 

BACKGROUND 

if 

30 

83 

RDGBKG20 

0 

051695 

READING 

BACKGROUND 

if 

31 

84 

RDGBKG20 

0 

095686 

READING 

BACKGROUND 

if 
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Table  C-17 


Estimated  Effects  for  Reading  Cross- sectional  Conditioning  Variables 
1988  Intercorrelation  Sample,  Grade  12/Age  17 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-0. 

285641 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

0. 

162409 

SEX  (FEMALE) 

3 

ETHNIC2 

-0. 

251546 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC3 

-0. 

329229 

OBf  ''ED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

-0. 

261376 

0'        ED  ETHNICITY  (ASIAN) 

6 

ST0C2 

0. 

074295 

SlZti  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0. 

041690 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

0. 

064445 

REGION  (SOUTHEAST) 

9 

IIEGI0N3 

0. 

165534 

REGION  (CENTR'-L) 

10 

REGI0N4 

0. 

082071 

REGION  (WEST) 

11 

PARF.D2 

-0. 

046125 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

-0. 

076116 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

-0. 

022725 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

-0 

279192 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

H0MEITM3 

0 

132520 

ARTICLES  IN  HOME  (YES  TO  3) 

16 

H0MEITM4 

0 

215761 

ARTICLES  IN  HOME  (YES  TO  4) 

17 

TV.  1 

0 

056234 

TV  WATCHING 

18 

TV.  2 

-0 

011638 

TV  WATCHING 

19 

HOMELNGl 

-0 

004110 

OTHER  LANGUAGE  AT  HOME  (SOMETIMES,  ALWAYS) 

20 

HW-NO 

-0 

377706 

HOMEWORK  (NONE  ASSIGNED) 

21 

HW-YES 

-0 

186437 

HOMEWORK  (YES  -  SOME  AMOUNT) 

22 

HW-2345 

0 

045292 

HOMEWORK  AMOUNT  (LINEAR) 

23 

2LUNCH  1 

0 

000698 

PERCENT  IN  LUNCH  PROGRAM 

24 

XLUNCH  2 

-0 

054536 

MISSING  LUNCH  PROGRAM 

25 

XWHITE  1 

-0 

.102933 

WHITE  MINORITY 

26 

XWHITE  2 

0 

.040726 

INTEGRATED 

27 

AGE/GRDl 

-0 

.457217 

MODAL  AGE,  <  MODAL  GRADE 

28 

AGE/GRD2 

-0 

.249898 

MODAL  AGE,  MODAL  GRADE,  MISSING 

29 

AGE/GRD4 

-0 

.502886 

>  MODAL  AGE,  MODAL  GRADE 

30 

SCH.TYPE 

-0 

.044550 

PUBLIC  V.  PRIVATE  SCHOOLS 

31 

H/HOMEWK 

-0 

.220694 

HELP  WITH  HOMEWORK  (EVERY  DAY,  TWICE  A  WEEK) 

32 

PAR/HOME 

-0 

.041477 

SINGLE/MULTIPLE  PARENT  (MULTIPLE) 

33 

MOTH/HOM 

0 

.010448 

MOTHER  AT  HOME  (YES) 

34 

MOTH/WRK 

0 

.001454 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

35 

PAGE/RDl 

0 

.217577 

PAGES  A  DAY  READ    (6  AND  UP) 

36 

PAGE/RD2 

-0 

.051548 

PAGES  A  DAY  READ     (11  AND  UP) 

37 

DAYSMISS 

-0 

.027542 

DAYS  OF  SCHOOL  MISSED  LAST  MONTH 

38 

GRADES 

0 

.283600 

GRADES  IN  SCHOOL 

39 

HS  PGM2 

0 

.133174 

HIGH  SCHOOL  PROGRAM  (COLLEGE  PREPARATORY) 

40 

HS  PGM3 

0 

.015758 

HIGH  SCHOOL  PROGRAM  v VOCATIONAL,  TECHNICAL) 

41 

P0STSEC2 

0 

.012074 

POST- SECONDARY  PLANS  (TWO-YEAR  COLLEGE) 

(continued) 
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Table  C-17  (continued) 


Esticated  Effects  for  Reading  Cross-sectional  Conditioning  Variables 
1988  Intercorrelation  Sample,  Grade  12/Age  17 


42 
43 
44 
45 


Variable 

POSTS EC 3 
WORKHOUR 
ENG. 23 
ENGLISH5 


Estimated 
Effect 

0.181729 
■0.001030 

0.103382 
-0.112947 


Description 

POST- SECONDARY  PLANS  (FOUR-YEAR  COLLEGE) 
HOURS  OF  OUTSIDE  WORK 

TYPE  OF  ENGLISH  CLASS (ADV.  PLACEMENT&COLLEGE) 
TYPE  OF  ENGLISH  CLASS (REMEDIAL) 


ERIC 


568 
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Table  C-18 


Estiipated  Effects  for  Reading  Cross-sectional  Conditioning  Variables 
1988  Grade  4  Students  with  Surveyed  Teachers 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-1. 

643733 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

0. 

054165 

SEX  (FEMALE) 

3 

ETHNIC2 

-0. 

170767 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC3 

-0. 

164307 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

-0. 

065474 

UBSERVED  ETHNICITY  (ASIAN) 

6 

ST0C2 

0. 

254828 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0. 

097182 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

0. 

113317 

REGION  (SOUTHEAST) 

9 

REGI0N3 

0. 

^^30617 

REGION  (CENTRAL) 

10 

REGI0N4 

0, 

030780 

REGION  (WEST) 

11 

PARED2 

0, 

229261 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0. 

295141 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

2664n 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

0 

2108b9 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

H0MEITM3 

0 

056898 

ARTICLES  IN  HOME  (YES  TO  3) 

16 

H0MEITM4 

0 

043555 

ARTICLES  IN  HOME  (YES  TO  4) 

17 

TV.  1 

0 

122396 

TV  WATCHING 

18 

TV.  2 

-0 

019522 

TV  WATCHING 

19 

HOMELNGl 

-0 

039185 

OTHER  LANGUAGE  AT  HOME  (SOMETIMES,  ALWAYS) 

20 

HW-NO 

0 

356544 

HOMEWORK  (NONE  ASSIGNED) 

21 

HW-YES 

0 

330480 

HOMEWORK  (YES  -  SOME  AMOUNT) 

22 

HW-2345 

0 

012380 

HOMEWORK  AMOUNT  (LINEAR) 

23 

XLUNCH  1 

-0 

001171 

PERCENT  IN  LUNCH  PROGRAM 

24 

ZLUNCH  2 

-0 

.170076 

MISSING  LUNCH  PROGRAM 

25 

XWHITE  1 

-0 

.013090 

WHITE  MINORITY 

26 

XWHITE  2 

-0 

.038655 

INTEGRATED 

27 

MA,MG 

-0 

.149529 

MODAL  AGE,  MODAL  GRADE,  MISSING 

28 

>MA,MG 

-0 

.221906 

>  MODAL  AGE,  MODAL  GRADE 

29 

SCH.TYPE 

0 

.108642 

PUBLIC  V.  PRIVATE  SCHOOLS 

30 

H/HOMEWK 

-0 

.033086 

HELP  WITH  HOMEWORK  (EVERY  DAY,  TWICE  A  WEEK) 

31 

PRESCH 

0 

.078302 

WENT  TO  PRESCHOOL 

32 

PAR/HOME 

-0 

.000269 

SINGLc/MULTIPLE  PARENT  (MULTIPLE) 

33 

MOTH/HOM 

0 

.152510 

MOTHER  AT  HOME  (YES) 

34 

MOTH/WRK 

-0 

.028558 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

35 

GRWN  HOM 

0 

.062530 

GROWNUP  AT  HOME  RIGHT  AFTER  SCHOOL 

36 

PAGE/RDl 

0 

.089515 

PAGES  A  DAY  READ     (6  AND  UP) 

37 

PAGE/RD2 

-0 

.018468 

PAGES  A  DAY  READ     (11  AND  UP) 

38 

RDGCOMPl 

0 

.066655 

READING  COMPOSITE  #  1 

39 

RDGC0MP2 

-0 

.003565 

REAI'ING  COMPOSITE  #  1 

40 

RDGBKGDl 

0 

.056316 

READING  BACKGROUND  //  1 

41 

RDGBKGD2 

-0 

.010363 

READING  B/iCKGROUND  //  2 

(continued) 
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Table  C-18  (continued) 


Estimated  Effects  for  Reading  Cross- sec tlom.l  Conditioning  Variables 
1988  Grade  4  Students  with  Surveyed  Teachers 


Estimated 


Variable 

Effect 

Description 

42 

IU)GBKGD3 

0 

011394 

RE/iDING  daCKGROUND  //  3 

43 

RDGBKGD4 

0 

033141 

READING  BACKGROUND  //  4 

44 

RDGBKGD5 

-0 

048777 

READING  BACKGROUND  //  5 

45 

RDGBKGD6 

0 

006944 

READING  BACKGROUND  //  6 

46 

RDGBKGD7 

-0 

021022 

READING  BACKGROUND  //  7 

47 

RDGBKGD8 

0 

021402 

READING  BACKGROUND  //  8 

48 

RDGBKGD9 

0 

138259 

READING  BACKGROUND  //  9 

49 

RDGBKGIO 

0 

019127 

READING  BACKGROUND  //  10 

50 

TQSl 

-0 

366611 

READING  LEVEL  OF  STUDENT 

51 

TQS3 

0.183620 

HIGH, AVERAGE, LOW  READING  LEVELS 

52 

TQS3_ 

-0 

113201 

HIGH, AVERAGE, LOW  READING  LEVELS(1 , 2 , 3) 

5j 

TQS4 

0 

045659 

HELP  STUDENT  WITH  READING  ALOUD 

54 

TQS5 

-0 

.040390 

HELP  STUDENT  WITH  COMPREHENSIVE  SKILLS 

55 

TQS6 

0 

.024487 

HELP  STODEN"^  WITH  WORD -ATTACK  SKILLS 

56 

TQS7 

0 

.004351 

HELP  STUDENT  WITH  VOCABULARY 

57 

TQSIO 

0 

.256632 

DOES  STUDENT  RECEIVE  REMEDIAL  READING  PROG 

58 

TQCl 

-0 

.016712 

ARE  STUDENTS  ASSIGNED  TO  CLASS  BY  ABILITY 

59 

TQC2 

-0 

.042102 

READING  ABILITY  LEVEL  OF  STUDENT  Hi  CLASS 

60 

TQC3 

-0 

.011451 

TIME  SPENT  ON  TYPICAL  DAY  FOR  DIRECT  INSTRUCT. 

61 

TQC4AYES 

-0 

.069730 

HAViS  READING  INSTRUCTION  -  INDIVIDUAL 

62 

TQC4AY1 

-0 

.007559 

HAVE  READING  INSTRUCHON  -  INDIVIDUAL(  L ,  2  ,  3 ,4) 

63 

TQC4B\^S 

-0 

.042395 

HAVE  READING  INSTRUCTION  -  SMALL  GROUP 

64 

TQC4BY2 

0 

.021101 

HAVE  READING  INSTRUCTION  -  SM.  GROUP (1,2, 3, 4) 

65 

TQC4CYFS 

0 

.022854 

HAVE  READING  INSTRUCTION  -  WHOLE  CLASS 

66 

TQ'-4CY2 

0 

.015939 

HAVE  READING  INSTRUCTION  -  WH.  CL/>SS(1 ,2, 3  ,4) 

67 

TQC6 

-0 

.023527 

ARE  STUDENTS  ASSIGNED  TO  GROUPS  BY  ABILITY 

68 

TQC7-YES 

-0 

.048101 

USE  SAME  READING  BOOK  FOP  ALL  IN  CLASS 

69 

TQC7-2 

0 

.022808 

USE  DIFFERENT  LEVELS  WITHIN  SAME  BASAL. 

70 

TQC7-3 

0 

.089372 

USE  DIFFERENT  BASAL  SERIES 

71 

TQC8A-8E 

0 

.003570 

NUMBER  OF  RESOURCES 

72 

TQCIO 

0 

.014164 

HOW  OFTEN  -  READ  ALOUD  TO  STUDENTS 

73 

TQCll 

0 

.000255 

HOW  OFTEN  -  HAVE  STUDENTS  MEET  IN  SMALL  GROUPS 

74 

TQC12 

-0 

.004808 

HOW  OFTEN  -  HAVE  STUD.  WRITE  ABOUT  SOMETHING 

75 

TQC13 

0 

.029973 

HOW  OFTEN  -  HAVE  STUDENT C  COMPLETE  WORKBOOKS 

76 

TQC14 

0 

.010835 

HOW  OFTEN  -  HAVE  STUD.  READ  BOOKS  THEY  CHOOSE 

77 

TQC15 

0 

.018556 

HOW  OFTEN  -  HAVE  STUD.  READ  INFORM.  MATERIAL 

7S 

TQC16 

-0 

.030612 

HOW  OFTEN  -  TAKE  CLASS  TO  LIBRARY 

79 

TQTB9 

0 

.009389 

HIGHEST  ACADEMIC  DEGREE  HELD 

80 

TQTB12BE 

0 

.035126 

N'TMBER  OF  SPECIAL  TRAININGS 

81 

TQTB23 

-0.001884 

ABILITY  TO  GET  DESIRED  INSTRUCTIONAL  MATERIALS 

82 

TQTB24 

-0 

.008206 

IF  '■•JULD  START  OVER,  WOULD  BECOME  A  TEACHER 

570 


Table  C-19 


Estimated  £tfects  for  Reading  Trend  Conditionii^g  Variables 
1988  Bridge  to  1984,  Age  9 


Estimated 


Variable 

Effect 

Descrtotion 

1 

OVERALL 

-1. 

105701 

OVERALL  CONSTANT    1    FOR  EVEP.YONE 

2 

GENDER2 

0. 

155957 

SEX  (FEMALEoc  MISSING) 

3 

ErHNIC2 

-0.410887 

ETHNICITY  (BLACK) 

4 

ETHNICS 

-0. 

297601 

ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

0, 

289799 

ETHNICITY  (ASIAN) 

6 

ST0C2 

0, 

173011 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

u , 

Uo JijU 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  J^W) 

8 

REGI0N2 

0, 

A1  O  ^  OO 

018628 

REGION  (SOUTHEAST) 

9 

REGIONS 

u , 

1  AAA01 
iUUUZi 

REGION  (CENTRAL) 

10 

REGI0N4 

AOQ  OQO 

Uyozoz 

REGION  (WEST) 

11 

PARED2 

0, 

O AO  O  ^  C 

202 J63 

PARENTS  EDUCATION  (dIGH  SCHOOL  GRAD; 

12 

PARED3 

0, 

1  OA  "7 OA 

iy4 / jU 

PARENTS  EDUCATION  (rOST  HIGH  SCHOOL) 

13 

PARED4 

0, 

0*7  "7  AOO 

2/ 7028 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

U , 

1  1  A 1AO 

ii4 jUj 

PARENTS  EDUCATION  (MISSING,   I  DON  T  KNOW) 

15 

ITEMS 

0, 

A"7  C  "7  C  "7 

075757 

ITEMS  IN  HOME 

16 

ITEMS2 

fx 

u 

OO  1  "7 1  O 

221/12 

ITEMS  IN  HOME  (BLANK  MISSING) 

1  "7 

17 

TV 

A  R  1  ATA 
Uji4 /4 

HOURS  TV  WATCHING  (LINEAR) 

1  o 

lo 

TV2 

OA/.7OS 
Z*+*+  /  ^  J 

HOURS  TV  WATCHING  (MISSING) 

1  n 
19 

nW-  lES 

-0 

022698 

HOMEWORK  (DUN  T  HAVE  ANY  ot  oUrlh  AWUUNi) 

OA 

20 

HW 

-0 

086636 

HOMEWORK  (MISSING) 

21 

IK  BY  Ei 

-0 

092054 

LANGUAGE  MINORITY  BY  ETHNICITY  (SPAN.  &  UTHbR^) 

22 

in 

-0 

.342136 

7  AXT0TTA017    1#T  XTAnTT^V    T>\7    T7T»TJXTT /^T»nV     /  K/f  T  C  C  T  X7l^  \ 

LANGUAGE  MINORITY  BY  ETHNICITY  (MlSblNG) 

Z  J 

0 

.124241 

24 

PAGE 

-0 

.305198 

PAGES  READ(MISSING) 

25 

LUNCHZ 

-0 

.189146 

PERCENT  IN  LUNCH  PROGRAM  (F3.2) 

26 

LUNCH 

0 

.041344 

LUNCH  PROGRAM  (MISSING) 

27 

%WHITE49 

-0 

.009154 

PERCENT  WHITE  IN  SCHOOL  (0-49X  WHITE  M'-JORITY) 

28 

2WHITE79 

0 

.058382 

PERCENT  WHITE  IN  SCHOOL  (50-79X  INTEGRATED) 

29 

XWHITEOO 

0 

.026256 

PERCENT  WHITE  IN  SCHOOL  (80  lOOZ  PREDOMINANTLY) 

30 

COURSES 7 

0 

.028015 

COURSES  TAKEN(l-7)  F4.1. 

31 

COURSES 

-0 

.017418 

COURSES  TAKEN(MISSING) 

32 

<MA,<MG 

-c 

.542511 

MODAL  AGE,  LESS  THAN  MODAL  GRADE 

33 

MA, KG 

-0 

.025866 

MODAL  AGE,  MODAL  GRADE,  MISSING 

34 

KA,>MG 

0 

.571595 

MODAL  AGE,  GREATER  THAN  MODAL  GRADE 

35 

>MA,MG 

-0 

.284144 

GREATER  THAN  MODAL  ACE,  MODAL  GRADE 

571 
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Table  C-20 


Estimated  Effects  for  Reading  Trend  Conditioning  Variables 
1988  Bridge  to  1984,  Age  13 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-0. 

514684 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

0. 

152055 

SEX  (FEHALE&  MISSING) 

3 

ETHNIC2 

-0. 

194921 

ETHNICITY  (BLACK) 

4 

ETHNIC3 

-0. 

171738 

ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

0. 

212954 

ETHNICITY  (ASIAN) 

6 

ST0C2 

0. 

184238 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0.123749 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

0. 

069277 

REGION  (SOUTHEAST) 

9 

REGIONS 

0. 

001563 

REGION  (CENTRAL) 

10 

REGI0N4 

0. 

020983 

REGION  (WEST) 

11 

PARED2 

0. 

048948 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0. 

214287 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0. 

205732 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED 

-0. 

016082 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

ITEMS 

0. 

057393 

ITEMS  IN  HOME 

16 

ITEMS2 

0. 

209762 

ITEMS  IN  HOME  (BLANK  MISSING) 

17 

TV 

-0. 

038498 

HOURS  TV  WATCHING  (LINEAR) 

18 

TV2 

-0. 

365324 

HOURS  TV  WATCHING  (MISSING) 

19 

HW-YES 

0. 

030688 

HOMEWORK  (DON'T  HAVE  ANY  &  SOME  AMOUNT) 

20 

HW 

-0. 

177569 

HOMEWORK  (MISSING) 

21 

m  BY  E3 

-0. 

007368 

lANGUAGE  MINORITY  BY  ETHKICITY  (SPAN.  &  OTHERS) 

22 

IM 

0. 

034672 

LANGUAGE  MINORITY  BY  ETHNICITY  (MISSING) 

23 

PAC£1234 

0. 

125767 

PAGES  READ (MORE  THAN  5) 

24 

PAGE 

-0. 

056604 

PAGES  READ (MIS SING) 

25 

LUNCHZ 

0. 

114286 

PERCENT  IN  LUNCH  PROGRAM  (F3.2) 

26 

LUNCH 

0. 

123994 

LUNCH  PROGRAM  (MISSING) 

27 

XWHITE49 

0. 

089640 

PERCENT  WHITE  IN  SCHOOL  (0-492  WHITE  MINORIT\) 

28 

XWHITE79 

0. 

224434 

PERCENT  WHITE  IN  .««HOOL  (50-79X  INTEGRATED) 

29 

XWHITEOO 

0. 

138460 

PERCENT  WHITE  IN  SCHOOL  (80-lOOX  PREDOMINANTLY) 

30 

COURSES 7 

0. 

018064 

COURSES  T.\KEN(l-7)  F4.1 

31 

COURSES 

-0. 

001403 

COURSES  TAKEN(MISSING) 

32 

<MA,<MG 

-0. 

368692 

MODAL  AGE,  LESS  THAN  MODAL  GRADE 

33 

MA.MG 

-0. 

020205 

MODAL  AGE,  MODAL  GRADE,  MISSING 

34 

MA,>MG 

0. 

083748 

MODAL  AGE,  GREATER  THAN  MODAL  GRADE 

35 

>MA,MG 

-0. 

211022 

GREATER  THAN  MODAL  AGE,  MODAL  GRADE 

ERIC 
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Table  C-21 


Estimated  Effects  for  Rea-j.ng  Trend  Conditioning  Variables 
1988  Bridge  to  1984,  Age  17 


Estimated 


Variable 

Effect 

np<5C"rtnt"i  on 

1 

OVERALL 

0 

197898 

OVERALL  CONSTANT  ' 1 '  FOR  FVFRYONF 

2 

FEMALE 

0 

078578 

SEX  rFEMALE&  MISSTN^'i 

3 

ST0C2 

-0 

309676 

SIZE  AND  TYPE  OF  COMMUNITY  ^HIGH  METRO 

4 

ST0C3 

-0 

144500 

SIZE  AND  TYPE  OF  COMMUNITY  ^NOT  HIGH  OR  LOW'i 

5 

REGI0N2 

0 

008773 

REGION  ^ SOUTHEAST) 

6 

REGI0N3 

0 

291371 

REGION  ^'CENTRAL'i 

7 

REGI0N4 

0 

194298 

REGION  fWEST') 

8 

PARED2 

-0 

020116 

PARENTS  EDUCATION  ^HIGH  <;CHO0T  GRAD'i 

9 

PARED3 

0 

011057 

PARENTS  EDUCATION  ^POST  HIGH  SCHOOL) 

X  *XXVfc»l  1  X           Xtft/ W            X  XwLl       \XWX      IIX  W  A  1      U  VilA  ^\J\J  S>^  1 

10 

PARED4 

-0 

023586 

PARENTS  EDUCATION  ^'COLLEGE  GR^^D) 

11 

PARED 

0 

063957 

PARENT'S  EDUCATION  ^'MISSING     I  DON'T  KNOW) 

12 

ITEMS^YES 

0 

253708 

ITEMS  IN  HOME 

X  X  XM  AU      XLl      A  IwA  AXi 

13 

ITEMS 

0 

262798 

ITEMS  IN  HOME  fBLANK  MISSING) 

14 

TV 

-0 

086053 

HOURS  TV  WATCHING  fLINEAR) 

15 

TV2 

0 

083831 

HOURS  TV  WATCHING  fMISSING) 

16 

HW-YES 

0 

075871 

HOMEWORK  (DON'T  HAVE  ANY  &  SOME  AMOUNT) 

4  AX^4  44^TT  X^A^A^       %  4^%^  41                 4  A4  *  V            4  44  «  A                  W         4^^      4  ^4  4^^  w41  ^  M 

17 

HW 

-0 

044334 

HOMEWORK  (M'^SSING) 

18 

LM23 

-0 

683796 

LANGUAGE  MINORITY  BY  ETHNICITY  (SPAN    &  OTHERS) 

^^444v^^  \y44^^           &  4^41  ^y4V^  JL  A             A             Jk  4  A41  ^       ^  JL  A        %  w  X  4441  «                       X  4  i^^4VW  M 

19 

LM_ 

0 

077812 

LANGUAGE  MINORITY  BY  ETHNICITY  (MISSING) 

20 

PAGE1234 

-0 

279886 

PAGES  READ(MORE  THAN  5) 

21 

PAGERD 

-0 

301479 

PAGES  READ(MISSING) 

22 

LUNCHX" 

-0 

550248 

PFJRCENT  IN  LUNCH  PROGRAM  (F3.2) 

23 

LUNCH 

0 

229743 

LUNCH  PROGRAM  (MISSING) 

24 

XWHITE49 

0 

518987 

PERCENT  WHITE  IN  SCHOOL  (0-49X  WHITE  MINORITY) 

25 

XWHITE79 

-0 

003205 

PERCENT  WHITE  IN  SCHOOL  (50-79Z  INTEGRATED) 

26 

XWHITEOO 

-0 

029302 

PERCENT  WHITE  IN  SCHOOL  (80-lOOX  PREDOMINANTLY; 

27 

ETHNIC2 

-0 

052682 

ETHNICITY  (BLACK) 

28 

ETHNIC 3 

-0 

033477 

ETHNICITY  (HISPANIC) 

29 

ETHNIC4 

-0 

069511 

ETHNICITY  (ASIAN) 

30 

<MA,<MG 

-0 

538537 

MODAL  AGE,  LESS  THAN  MODAL  GRADE 

31 

MA.MG 

-0 

113926 

MODAL  AGE,  MODAL  GRADE,  MISSING 

32 

MA,:^G 

0 

065530 

MODAL  AGE,  GREATER  THAN  MODAL  GRADE 

33 

^,MG 

-0 

458032 

GREATER  THAN  MODAL  AGE,  MODAL  GRADE 

Table  C-22 


Estimated  Effects  for  Civics  Cross -sectional  Conditioning  Variables 
1988  Focused- BIB  Sample,  Grade  4/Age  9 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-0, 

862717 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER- F 

-0. 

023548 

SEX  (FEMALE) 

3 

ETHNIC -B 

-0. 

212695 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC -H 

-0. 

250943 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC -A 

-0. 

151950 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

STOC-H 

0. 

223446 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOC-X 

0. 

097596 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REG-SE 

0. 

045772 

REGION  (SOUTHEAST) 

9 

REG-C 

0. 

024092 

REGION  (CliNTRAL) 

10 

REG-W 

-0. 

040014 

REGION  (WEST) 

11 

PAREDHS 

0. 

010546 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

0. 

090901 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDC 

0. 

108382 

PARENTS  EDUCATION  (COLLEGE  GRADUATE) 

U 

PAREDH 

-0. 

089029 

PARENTS  EDUCATION  (MISSINu,  I  DON'T  WWW) 

15 

ITEMS -3 

0. 

110087 

3  ITEMS  IN  THE  HOME 

16 

ITEMS -4 

0. 

180746 

4  ITEMS  IN  THE  HOME 

17 

TVWATCH 

0. 

118004 

NO.  HOURS  OF  TV  WATCHED  PER  DAY 

18 

TVWATCH2 

-0. 

018143 

NO.  HOURS  SQUARED  OF  TV  WATCHED  PER  DAY 

19 

HOMELANG 

0. 

044591 

MINORITY  HOME  LANGUAGE  SPOKEN 

20 

HOMEWRK- 

-0. 

348922 

DON'T  HAVE  HOMEWORK 

21 

HOMEWRKO 

-0. 

367229 

DON'T  DO  HOMEWORK 

22 

HOMEWORK 

-0. 

010127 

AMOUNT  OF  HOMEWORK  DONE 

23 

FCTLUNCH 

-0. 

001857 

PERCENT  IN  LUNCH  PROGRAM 

24 

NOLUNCH 

-0. 

049798 

DON'T  HAVE  LUNCH  PROGRAM  OR  MISSING 

25 

PCTWHLOW 

-0. 

062140 

PCT  WHITE  IN  SCHOOL  0-49 

26 

PCTWHMED 

0. 

019981 

PCT  WHITE  IN  SCHOOL  50-79 

27 

-MA:<MG 

-0. 

657937 

AT  MODAL  AGE;  BELOW  MODAL  GRADE 

2c 

-MA:-MG 

-0. 

206925 

AT  MODAL  AGE;  AT  MODAL  GRADE 

2, 

■'MA:>MG 

0. 

389979 

AT  MODAL  AGE;  ABOVE  MODAL  GRADE 

30 

>MA:-MG 

-0. 

305617 

ABOVE  MODAL  AGE;  AT  MODAL  GRADE 

31 

SCH-PRIV 

0. 

089136 

SCHOOL  OTHER  THAN  PUBLIC 

32 

HOMEHELP 

-0. 

110131 

USUALLY  GET  HELP  AT  HOME  WITH  HOMEWORK 

33 

PRESCH 

0. 

065690 

WENT  TO  PRESCHOOL 

34 

HOMEPARS 

0. 

066022 

FATHER  AND  MOTHER  BOTH  AT  HOME 

35 

HOMEMOM 

0. 

219030 

MOTHER  LIVES  AT  HOME 

36 

MOMWORK 

0. 

036292 

MOTHER  WORKS  OUTSIDE  OF  HOME 

37 

HOMEGRO 

0. 

037314 

GROWNUP  AT  HOME  RIGHT  AFTER  SCHOOL 

38 

RD06+PP 

0. 

034761 

READ  6+  PAGES  PER  DAY  FOR  SCHOOL 

39 

RDII+PP 

0. 

065288 

READ  11+  PAGES  PER  DAY  FOR  SCHOOL 

40 

SSEVERY 

0. 

101641 

SOCIAL  STUDIES  CLASS  EVERY  DAY 

41 

SS3-4 

0. 

163122 

SOCIAL  STUDIES  CLASS  3-4  TIMES  PER  WEEK 

(continued) 
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Table  C-22  (continued) 


Estimated  Effects  for  Civics  Cross- sectional  Conditioning  Variables 
1988  Focused- BIB  Sample,  Grade  4/Age  9 


Estimated 


Variable 

Effect 

Description 

42 

SSI -2 

0. 

093823 

SOCIAL  STUDIES  CLASS  1-2  TIMES  PER  WEEK 

43 

SS<1 

0. 

038207 

SOCIAL  STUDIES  CLASS  <  1  TIME    PER  WEEK 

44 

STGOV 

0. 

012140 

STUDIED  COVERl^MENT 

45 

STGOV-X 

-0. 

192139 

STUDIED  GOVERNMENT  RESP.  MISSING 

46 

ST-ALOT 

-0. 

019732 

NUMBER  OF  SUBJECTS  STUDIED  A  LOT  (OUT  OF  6) 

47 

STLAW-L 

-0. 

044790 

STUDY  LAWS  A  LOT  OR  SOME 

48 

STCRT-L 

-0. 

026685 

STUDY  JUDGES  &  COURTS  A  LOT  OK.  SOM£ 

49 

STPRES-L 

-0. 

003562 

STUDY  PRES.  &  LEADERS  A  LOT  OR  SOME 

50 

STVOT-L 

0. 

059727 

STUDY  VOTING  &  ELECTIONS  ^  LOT  OB  SOME 

51 

STCOMM-L 

0 

081548 

STUDY  COMMUNITY  A  LOT  OR  SOME 

52 

STRGHT-L 

0. 

092128 

STUDY  RIGHTS  6.  RESPONSIBILITIES  A  LOT  OR  SOME 

53 

CE- EVERY 

-0 

050833 

DISCUSS  CURRENT  EVENTS  AUJDST  EVERY  DAY 

54 

GEl-2 

0 

049644 

DISCUSS  CURRENT  EVENTS  1-2  TIMES  A  WEEK 

55 

CE<1 

0 

.073888 

DISCUSS  CURRENT  EVENTS  1-2  TIMES  A  MONTH 

56 

CE«1 

-0 

022439 

DISCUSS  CURRENT  EVENTS  FEW  TIMES  A  YEAR 

575 
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Table  C-23 


Estimated  Effects  for  Civics  Cross-sectional  Conditioning  Variables 
1988  Intercorrelation  Sample,  Grade  4/Age  9 


Estimated 


Variable 

Dp  <5p"r'f  nt"i  nn 

1>C  OV«X  XL/  L.XU11 

1 

OVERALT 

-2 

423182 

OVFRAT  T    rONSTANT  ' 1  '    FOR  FVFRYONF 

\^VdC\AX«X«    Ox^ilO  Xnii  X        X        k\JK^        V  I^IxX  \/il 

2 

-0 

02QS71 

SEX  ^ FEMALES 

3 

ORSFR't/FH  FTHNiriTY  ^RTAPK^ 

4 

FTHNTT-H 

-0 

£..J  JLSJ  JL  7 

ORSFRVFH  FTHMiriTY  ^HTSPANir^ 

5 

FTHNTT-A 

0 

ORSFRVFH  FTHNiriTY  ^ASTAN  AMFRTPAKf^ 

6 

STOfl-H 

0 

ST7F  AND  TYPE  OF  rOMMTINTTY  ^HTGH  METROS 

7 

«J  X  ^ 

0 

X  XV/-'  ^  u 

SIZE  AND  TYPE  OF  COMMUNITY  ^NOT  HIGH  OR  IjOU^ 

i7X£«U    mil/     XXXXj     \/X      v\/lU^VIilXXX      ^il\/X     11XV711     \/l\     LAJYI  / 

8 

REG-SE 

0 

110936 

REGION  ^ SOUTHEAST^ 

CVUUXV/il      ^i/\/U  Xkl&rfX\k/  X  / 

9 

REG-C 

0 

090622 

REGION  ^CENTRA! ^ 

10 

REG-W 

-0 

003717 

REGION  ^UFST^ 

11 

PARFDHS 

0 

U     U  V 

PARFNTS  FHTTPATION   ^HIGH  STHOOT  GRAD^ 

12 

0 

9Q8Q'^9 

PARENTS  FHTTPATION   ^POST  HI^^H  SPHOOT  ^ 

13 

PARE DC 

0 

272740 

PARENTS  FDIIPATION  ^COTTEGE  GRADUATE^ 
xruvciiiXtj    CiL/uonxxuii    \\j\Ji^tMLu\jLu   V7i\xvi/\inx  xj  / 

14 

PAREDM 

0 

049526 

PARENTS  EDUCATION  ^MISSING     I  DON'T  KNOU^ 

15 

ITEMS -3 

0 

052185 

3  ITEMS  IN  THE  HOME 

J     XX  Cm  ItJ     Xl«     X 1 11^    llx/i  lU 

16 

ITEMS -4 

0 

141589 

4  ITFJ4S  IN  THE  HOME 

^      X  X        liJ      X  &1      X 1  lu     la\/l  lU 

17 

TVUATCH 

X  V  WV«A  X  wit 

0 

277411 

NO    HOURS  OF  TV  WATCHED  PER  DAY 

18 

TVWATCH2 

-0 

037488 

NO    HOURS  SQUARED  OF  TV  WATCHED  PER  DAY 

19 

HOMELANG 

-0 

021578 

MINORITY  HOME  LANGUAGE  SPOKEN 

20 

HOMEWRK- 

0 

312557 

DON'T  HAVE  HOMEWORK 

21 

HOMEWRKO 

0 

308734 

DON'T  DO  HOMEWORK 

22 

HOMEWORK 

1  i\/l  XXJ  If  \/ 1\X\ 

-0 

010786 

AMOUNT  OF  HOMEWORK  DONE 

m  Ix/  Ull  X     x/X      llx/i  lUWx/lv£\     I/x/11  u 

23 

PCTLUNCH 

-0 

002942 

PERCENT  IN  LUNCH  PROGRAM 

24 

NOLUNCH 

-0 

108688 

DON'T  HAVE  LUNCH  PROGRAM  OR  MISSING 

25 

PCTWHLOU 

-0 

094621 

PCT  WHITE  IN  SCHOOT  0-49 

X  wX     WXlXXu     XM     k7\JllV/\/X^    w     ~T  ^ 

26 

PCTWHMED 

•0 

015439 

PCT  WHITE  IN  SCHOOL  50-79 

X^X          nllXXlal          ^Jk^                \Jkl\J\J  X^        ai/Xy          /  > 

27 

-MA:<MG 

-0 

017183 

AT  MODAT   AGE'  BELOW  MODAL  GRADE 

28 

'   \ :  »'MG 

0 

421162 

AT  MODAT   AGE'  AT  MODAT  GRADE 

29 

-  ,A:>MG 

0 

458405 

AT  MODAL  AGE;  ABOVE  MODAL  GRADE 

30 

::^:-MG 

0 

351086 

ABOVE  MODAL  AGE;  AT  MODAL  GRADE 

31 

SCH-PRIV 

0 

001673 

SCHOOL  OTHER  THAN  PUBLIC 

32 

HOMEHELP 

-0 

104533 

USUALLY  GET  HELP  AT  HOME  WITH  HOMEWORK 

33 

PRESCH 

0 

104314 

WENT  TO  PRESCHOOL 

36 

HOMEPARS 

0 

033862 

FATHER  AND  MOTHER  BOTH  AT  HOME 

35 

HOMEMOM 

0 

110458 

MOTHER  LIVES  AT  HOME 

36 

MOMWORK 

0 

011762 

MOTHER  WORKS  OUTSIDE  OF  HOME 

37 

HOMEGRO 

0 

.058718 

GROWNUP  AT  HOME  RIGHT  AFTER  SCHOOL 

38 

RD06+PP 

0 

.070232 

READ  6+  PAGES  PER  DAY  FOR  SCHOOL 

39 

RDll+PP 

0 

030869 

READ  11+  PAGES  PER  DAY  FOR  SCHOOL 

40 

SSEVERY 

0 

.146238 

SOCIAL  STUDIES  CLASS  EVERY  DAY 

41 

SS3-4 

0 

201378 

SOCIAL  STUDIES  CLASS  3-4  TIMES  PER  WEEK 

( continued) 
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Table  C-23  (continued) 

Estimated  Effects  for  Civics  Cross- sectional  Conditioning  Variables 
1988  Intercorrelat-.ion  Sample,  Grade  4/Age  9 


Estimated 


y,Tiable 

Effect 

Description 

42 

SSl-2 

0.232414 

SOCIAL  STUDIES  CLAS^  1-2 

TIMES 

PER  WEEK 

43 

SS<1 

0.060544 

SOCIAL  STUDIES  CLASS  <  1 

TIME 

.?ER  WEEK 

44 

STGOV 

0.033795 

STUDIED  GOVERNMENT 

45 

STGOV-X 

-0.500008 

STUDIED  GOVERNMENT  RESP. 

MISSING 

46 

STLAW-L 

0.068436 

STUDY  LAWS  A  LOT  OR  SOME 

47 

STPRES-L 

0. 007628 

STUDY  PRES.  &  LEADERS  A  LOT  OR 

SOME 

48 

STCOMM-L 

-0.074725 

STUDY  COMMUNITY  A  LOT  OR 

SOME 

ERIC 
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Table  C-24 


Estimated  Effects  for  Civics  Cross -sectional  Conditioning  Variables 
1988  Focused-BIB  Sample,  Grade  8/Age  13 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-1 

613136 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER- F 

-0 

077074 

SEX  (FEMALE) 

3 

ETHNIC -B 

-0 

138794 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC -H 

-0 

183800 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC-A 

-0 

101094 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

STOC-H 

0 

114793 

SIZE  AND  TYPE  OF  COMMUNir:  (HIGH  METRO) 

7 

STOC-X 

0 

061629 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REG-SE 

0 

025581 

REGION  (SOUTHEAST) 

9 

REG-C 

0 

071906 

REGION  (CENTRAL) 

10 

REG-W 

0 

014446 

REGION  (WEST) 

11 

PAREDHS 

0 

015981 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

0 

125404 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDC 

0 

154213 

PARENTS  EDUCATION  (COLLEGE  GRADUATE) 

14 

PAREDM 

-0 

081084 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

ITEMS -3 

0 

032833 

3  ITEMS  IN  THE  HOME 

16 

ITEMS -4 

0 

101106 

4  ITEMS  IN  THE  HOME 

17 

TVWATCH 

0 

046246 

NO.  HOURS  OF  TV  WATCHED  P  :R  DAY 

18 

TVWATCH2 

-0 

008183 

NO.  HOURS  SQUARED  OF  TV  WATCHED  PER  DAY 

19 

HOMELANG 

0 

007020 

MINORITY  HOME  LANGUAGE  SPOKEN 

20 

HOMEWRK- 

-0 

061921 

DON'T  HAVE  HOMEWORK 

21 

HOMEWRKO 

0 

026918 

DON'T  DO  HOMEWORK 

22 

HOMEWORK 

0 

007356 

AMOUNT  OF  HOMEWORK  DONE 

23 

PCT^JUNCH 

-0 

001397 

PERCENT  IN  LUNCH  PROGRAM 

24 

NOLUNCH 

-0 

090389 

DON'T  HAVE  LUNCH  PROGRAM  OR  MISSING 

25 

PCTWHLOW 

-0 

022594 

PCT  WHITE  IN  SCHOOL  0-49 

26 

PCTWHMED 

-0 

041061 

PCT  WHITE  IN  SCHOOL  50-79 

27 

-MA:<MG 

-0 

179729 

AT  MODAL  AGE;  BELOW  MODAL  GRADE 

28 

-MA:-MG 

0 

032173 

AT  MODAL  AGE;  AT  MODAL  GRADE 

29 

-MA:>MG 

0 

407612 

AT  MODAL  AGE;  ABOVE  MODAL  GRADE 

30 

>MA:-MG 

-0 

124894 

ABOVE  MODAL  AGE;  AT  MODAL  GRADE 

31 

SCH-PRIV 

0 

Ool854 

SCHOOL  OTHER  THAN  PUBLIC 

32 

HOMEHELP 

-0 

088763 

USUALLY  GET  HELP  AT  HOME  WITH  HOMEWORK 

33 

HOMEPARS 

0 

014917 

FATHEX  AND  MOTHER  BOTH  AT  HOME 

34 

HOMEMOM 

0 

15867f 

MOTHER  LIVES  AT  HOME 

35 

MOMWORK 

0. 

00681. 

MOTHER  WORKS  OUTSIDE  OF  HOME 

36 

RD06+PP 

0. 

105422 

READ  6+  PAGES  PER  DAY  FOR  SCHOOL 

37 

RDIUPP 

-0. 

011569 

READ  11+  PAGES  PER  DAY  FOR  SCHOOL 

'^8 

GRADEXP 

0 

207459 

EXPECT  TO  GRADUATE  FROM  HICH  SCHOOL 

39 

NOMISS 

0. 

007670 

DID  NOT  MISS  MUCH  SCHOOL  LAST  IIONTH 

40 

GRADES 

0. 

196147 

GRADES  IN  SCHOOL 

41 

STCV-INC 

-0. 

049973 

STUDIED  CIVICS  -  INCOMPLETE  SET  OF  ANSWERS 

(continued) 
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Table  C-24  (continued) 


Estimated  Effects  for  Civics  Cross-sectional  Conditioning  Variables 
1988  Focused- BIB  Sample    Grade  8/Age 


Estimated 


Variable 

Effect 

Descrlotlon 

42 

STCIV5 

-0. 

022543 

STUDIED  CIVICS  IN  5TH  GRADE 

43 

STCIV6 

-0 

022328 

STUDIED  CIVICS  IN  6TH  GRADE 

^  A  \Ji^  Jim  4^  4y      VfJkVJkV^       JL  41      Vy  A  4  4      V7  4\a  1 4^ 

44 

STCIV7 

-0 

034185 

STUDIED  CIVICS  IN  7TH  GRADE 

45 

STCIV8 

0 

140325 

STUDIED  CIVICS  IN  8TH  ^  *ADE 

46 

ST-ALOT 

0 

005388 

V  V  «f/  «y  V  V 

NUMBER  OF  SUBJECTS  ST  #"  i.ED  A  LOT  fOUT  OK  iU) 

4«  V/4  4A^4^4X      V  4             wAi^^  4afV<'  ^  %J       ^  4  ^    r     A  4^4i/     44      4^V^  X       \             X       W4        Jtr\I  J 

47 

STCON-L 

0 

154097 

SITIDIED  U  S     CONSTITUTION  A  LOT  OR  SOME 

JL            Jk  4af*/      W  •         •        W/41  ^X  JkXVXAv4l      44      4^V^  X       \/4X      ^\/4  44J 

48 

STCONG-L 

0 

029228 

STUDIED  CONGRESS  A  LOT  OR  SOME 

49 

0 

072744 

STUniFD  TAU^  A  TjOT  OR  SOMF 

50 

STCRT-L 

0 

111552 

STUDIED  COURTS  A  LOT  OR  SOME 

51 

STPRES-L 

0 

050512 

STUDIED  PRESIDENT  A  LOT  OR  SOME 

52 

STPOL-L 

-0 

011655 

STUDIED  POLITICAL  PARTIES  A  LOT  OR  SOME 

53 

STSLG-L 

0 

000487 

STUDIED  STATE/LOCAL  GOVS.  A  LOT  OR  SOME 

54 

STDG-L 

-0 

021626 

STUDIED  DEMOCRATIC  GOVS.  A  LOT  OR  SOME 

55 

STOG-L 

-0 

00672C 

STUDIED  OTHER  GOVS.  A  LOT  OR  SOME 

56 

STRGHT-L 

-P 

013451 

STUDIED  RIGHTS  &  RESPONSTB.  A  LOT  OR  SOME 

57 

CVA-AUT 

-0 

027045 

ACTIVE  IN  4  AREAS  OF  CIVICS  (1,2,5,8) 

58 

CVB^ACT 

0 

'^-•^976 

ACTIVE  IN  2  AREAS  OF  CIVICS  (3,  10) 

59 

CVC-ACT 

0 

-  ,165 

ACTIVE  IN  2  AREA?  OF  CIVICS  (4,  9) 

60 

CVD-ACT 

-J 

383355 

ACTIVE  IN  2  AREAS  OF  CIV.CS  (6  7) 

61 

CVD-NOT 

0 

147750 

NOT  VERY  ACTIVE  IN  THESE  2  AREAS  OF  CIVICS 

62 

RDDIF 

-0 

227011 

AT  LEAST  SOME  DlfFICULT/  IN  READIwG  OR  MISSING 

ERIC 
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Table  C-25 


Estimated  Effects  for  Civics  Cross-sectional  Conditioning  Variables 
1988  Intercorrelation  Sample,  Grade  8/Age  13 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-1 

224095 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER-F 

-0 

118153 

SEX  (FEMALE) 

3 

ETHNIC -B 

-0 

135472 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC -H 

-0 

193761 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC -A 

0 

027554 

OBSERVED  ETHNICITY  (ASIAN  AiMERICAN) 

6 

STOC-H 

0 

080959 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOC-X 

0 

046864 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REG-SE 

-0 

001454 

REGION  (SOUTHEAST) 

9 

REG-C 

-0 

001889 

REGION  (CENTRAL) 

10 

REG-W 

-0 

055975 

REGION  (WEST) 

11 

PAREDHS 

0 

030488 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

0 

128680 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDC 

0 

151727 

PARENTS  EDUCATION  (COLLEGE  GRADUATE) 

14 

PAREDM 

-0 

068832 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

ITEMS -3 

0 

122664 

3  ITEMS  IN  THE  HOME 

16 

ITEMS -4 

0 

.173642 

4  ITEMS  IN  THE  HOME 

17 

TVWATCH 

0 

020052 

NO.  HOURS  OF  TV  WATCHED  PER  DAY 

18 

TVWATCH2 

-0 

006978 

NO.  HOURS  SQUARED  OF  TV  WATCHED  PER  DAY 

19 

HOMELANG 

-0 

046921 

MINORITY  HOME  LANGUAGE  SPOKEN 

20 

HOMEWRK- 

0 

094715 

DON'T  HAVE  HOMEWORK 

21 

HOMEWRKO 

0 

205456 

DON'T  DO  HOMEWORK 

22 

HOMEWORK 

0 

040729 

AMOUNT  OF  HOMEWORK  DONE 

23 

PCTLUNCH 

-0 

001694 

PERCENT  IN  LUNCH  PROGRAM 

24 

NOLUNCH 

-0 

048478 

DON'T  HAVE  LUNCH  PROGRAM  OR  MISSING 

25 

PCTWHLOW 

0 

019132 

PCT  WHITE  IN  SCHOOL  0-49 

26 

PCTWHMED 

-0 

017506 

PCT  WHITE  IN  SCHOOL  50-79 

27 

-MA:<MG 

-0 

307013 

AT  MODAL  AGE;  BELOW  MODAL  GRADE 

28 

-MA:-MG 

-0 

008815 

AT  MODAL  AGE;  AT  MODAL  GRADE 

29 

-MA:>MG 

0 

332372 

AT  MODAL  AGP".;  AB''  'E  MODAL  GRADE 

30 

>MA:-MG 

-0 

127282 

ABOVE  MODAL  AGE;  AT  MODAL  GRADE 

31 

SCH-PRIV 

0 

160774 

SCHOOL  OTHER  THAN  PUBLIC 

32 

HOMEHELP 

-0 

134069 

USUALLY  GET  HELP  AT  HOME  WITH  HOMEWORK 

33 

HOHEPARS 

-0 

009036 

FATHER  AND  MOTHER  BOTH  AT  HOME 

34 

HOMEMOM 

0 

1233 A4 

MOTHER  LIVES  AT  HOME 

35 

MOMWORK 

-0 

022786 

MOTHER  WORKS  OUTSIDE  OF  HOME 

36 

RD06+PP 

0 

096868 

READ  6+  PAGES  PER  DAY  FOR  SCHOOL 

37 

RDll+PP 

0 

028811 

READ  11+  PAGES  PER  DAY  FOR  SCHOOL 

38 

GRADEXP 

0 

034207 

EXPECT  TO  GRADUATE  FROM  HIGH  SCHOOL 

39 

NOMISS 

0 

018810 

DID  NOT  MISS  MUCH  SCHOOL  LAST  MONTH 

40 

GRADES 

0 

283918 

GRADES  IN  SCHOOL 

41 

STCV-INC 

-0 

061693 

STUDIED  CIVICS  -  INCOMPLETE  SET  OF  ANSWERS 

(continued) 
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Table  C-25  (continued) 


Estimated  Effects  for  Civics  Cross -sectional  Conditioning  Variables 
1988  Intercorrelation  Sample,  Grade  8/Age  13 


Estimated 


Variable 

Effect 

Description 

42 

STCIV5 

-0 

068930 

STUDIED 

CIVICS  IN  y  SHADE 

43 

STCIV6 

0 

011925 

STUDIED 

CIVICS  IN  (  3RADE 

44 

STCIV7 

-0 

.031176 

STUDIED 

CIVICS  IN  7TH  GRADE 

45 

STCIV8 

0 

.039059 

STUDIED 

CIVICS  IN  8TH  GRADE 

46 

STCON-L 

0 

.002768 

STUDIED 

U.S.  CONSTITUTION  A  LOT 

OR 

SOME 

47 

STCONG-L 

0 

.094652 

STUDIED 

CONGRESS  A  LOT  OR  SOME 

48 

STLAW-L 

-0 

.051939 

STUDIED 

LAWS  A  LOT  OR  SOME 

49 

STCRT-L 

0 

.100526 

STUDIED 

COURTS  A  LOT  OR  SOME 

50 

STPRES-L 

-0 

.039286 

STUDIED 

PRESIDENT  A  LOT  OR  SOME 

51 

STPOL-L 

0 

.054987 

STUDIED 

POLITICAL  PARTIES  A  LOT 

OR 

SOME 

52 

STSLG-L 

0 

. 000008 

STUDIED 

STATE/LOCAL  GOVS.  A  LOT 

OR 

SOME 

53 

STDG-L 

-0 

.014294 

STUDIED 

DEMOCRATIC  GOVS.  A  LOT  OR  SOME 

620 


Table  C-26 


Estimated  Effects  for  Civics  Cross-sectional  Conditioning  Variables 
1988  Focused-BIB  Sample,  Grade  12/Age  17 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-1 

.329683 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER-F 

-0 

.161951 

SEX  (FEMALE) 

3 

ETHNIC -B 

-0 

.225/' 27 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC-H 

-0 

.123948 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC-A 

0 

.037675 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

STCC-H 

0 

.175198 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOC-X 

0 

.061377 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REG-SE 

-0 

.004221 

REGION  (SOUTHEAST) 

9 

REG-C 

0 

.061998 

REGION  (CENTRAL) 

10 

REG-W 

-0 

.018295 

REGION  (WEST) 

11 

PAREDHS 

0 

.024432 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

0 

.150551 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDC 

0 

.129662 

PARENTS  EDUCATION  (COLLEGE  GRADUATE) 

14 

PAREDM 

-0 

.029806 

PARENTS  EDUCATION  (MISSING.  I  DON'T  KNOW) 

15 

ITEMS -3 

0 

.144291 

3  ITEMS  IN  THE  HOME 

16 

ITEMS -4 

0 

.150336 

4  ITEMS  IN  THE  HOME 

17 

TVWATCH 

0 

.062647 

NO.  HOURS  OF  TV  WATCHED  PEu  DAY 

18 

TVWATCH2 

-0 

.012239 

NO.  HOURS  SQUARED  OF  TV  WATCHED  PER  DAY 

19 

HOMELANG 

-0 

.078055 

MINORITY  HOME  LANGUAGE  SPOKEN 

20 

HOMEWRK- 

-0 

007298 

DON'T  HAVE  HOMEWORK 

21 

HOMEWRKO 

0 

089084 

DON'T  DO  HOMEWORK 

22 

HOMEWORK 

-0 

001492 

AMOUNT  OF  HOMEWORK  DONE 

23 

PCTLUNCH 

0 

00030? 

PERCENT  IN  LUNCH  PROGRAM 

24 

NOLUNCH 

-0 

004135 

DON'T  HAVE  LUNCH  PROGRAM  OR  MISSING 

25 

PCTWHLOW 

-0 

011627 

PCT  WHITE  IN  SCHOOL  0-49 

26 

PCTWHMED 

0 

012093 

PCT  WHITE  IN  SCHOOL  50-79 

27 

-MA:<MG 

-0 

089241 

AT  MODAL  AGE;  BELOW  MODAL  GRADE 

28 

-MA:-MG 

-0 

017791 

AT  MODAL  AGE;  AT  MODAL  GRADE 

29 

>MA:-MG 

-0 

066792 

ABOVE  MODAL  AGE;  AT  MODAL  GRADE 

30 

SCH-PRIV 

-0 

027147 

SCHOOL  OTHER  THAN  PUBLIC 

31 

HOMEHELP 

-0 

185448 

USUALLY  GET  HELP  AT  HOME  WITH  HOMEWORK 

32 

HOMEPARS 

-0 

002427 

FATHER  AND  MOTHER  BOTH  AT  HOME 

33 

HOMEMOM 

0 

132333 

MOTHER  LIVES  AT  HOME 

34 

MOMWORK 

0 

021772 

MOTHER  WORKS  OUTSIDE  OF  HOME 

35 

RD06+PP 

0 

059618 

READ  6-f  PAGES  PER  DAY  FOR  SCHOOL 

36 

RD11-*PP 

0 

008460 

READ  11+  PAGES  PER  DAY  FOR  SCHOOL 

37 

NOMISS 

0 

022331 

DID  NOT  MTSS  MUCH  SCHOOL  LAST  MONTu 

38 

GRADES 

0 

120805 

GRADES  IN  SCHOOL 

39 

COLLPREP 

0 

117544 

COLLEGE  PREP.  PROGRAM 

40 

VOCTECH 

-0 

057613 

VOCATIONAL/TECH.  PROGRAM 

41 

PLAN2YR 

0 

016887 

PL\N  TO  GO  TO  2 -YEAR  COLLEGE 

(continued) 
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Tabl^  C-26  (continued) 


Estimated  Effects  for  Civics  Cross-sectional  Conditioning  Variables 
^988  Focused-BIB  Sample,  Grade  12/Age  17 


Estimated 


Variable 

Effect 

Description 

42 

PLAN4YR 

0. 

151381 

PLAN  TO  GO  TO  4 -YEAR  COLLEGE 

43 

WORKHRS 

-0. 

001538 

NO.  HOURS  OF  OUTSIDE  WORK 

44 

ENG-AP 

0. 

097595 

TAKING  A. P. /COLL.  PREP.  ENGLISH  CLASS 

45 

ENG-REM 

-0. 

195803 

TAKING  REMEDIAL  ENGLISH  CLASS 

46 

STCV-INC 

-0. 

148636 

STUDIED  CIVICS  -  INCOMPLETE  SET  OF  ANSWERS 

47 

STCV9 

-0. 

063248 

STUDIED  CIVICS  IN  9TH  GRADE 

48 

STCVIO 

-0. 

089337 

STUDIED  CIVICS  IN  lOTH  GRADE 

49 

STCVll 

-0. 

071126 

STUDIED  CIVICS  IN  IITH  GRADE 

50 

STCV12 

0 

037421 

STUDIED  CIVICS  IN  12TH  GRADE 

51 

GOVT 

0 

044203 

HOW  MANY  YEARS  OF  GOV'T  STUDY  HAVE  YOU  HAD 

52 

AP 

-0 

109859 

ADVANCED  PLACEMENT  YES 

53 

ST-ALOT 

0 

000510 

NUMBER  OF  SUBJECTS  STUDIED  A  LOT  (OUT  OF  10) 

54 

ST-AVE 

0 

364440 

AVERAGE  LEVEL  OF  STUDY  FOR  THESE  TEN 

55 

CIV- INT 

0 

067754 

LEVEL  OF  INTEREST  IN  CIVICS 

56 

MOCK-SOM 

0 

083116 

MOCK  ELECTIONS,  TRIALS,  ONCE  TO  SEVERAL  TIMES 

57 

GOVATTN 

0 

110814 

DEGREE  OF  ATTENTN  GOVT.  PAYS  (AVE.  OF  2  ITEMS) 

58 

HADCIV 

-0 

025435 

YES,  HAVE  HAD  A  CIVICS  CLASS 

59 

CVHWORK 

0 

032976 

AMOUNT  OF  CIVICS  HOMEWORK 

60 

CVGRADES 

0 

121410 

CIVICS  GRADES  IN  SCHOOL 

61 

CVA-ACT 

-0 

045429 

ACTIVE  IN  4  AREAS  OF  CIVICS  (1,2,5,8) 

62 

CVB-ACT 

-0 

069698 

ACTIVE  IN  2  AREAS  OF  ClVIfS  (3,  10) 

63 

CVC-ACT 

0 

089997 

ACTIVE  IN  2  AREAS  OF  CIVICS  (4,  9) 

64 

CVD-ACT 

-0 

193582 

ACTIVE  IN  2  APEAS  OF  CIVICS  (6,  7) 

65 

CVD-NOT 

0 

071695 

NOT  VERY  ACTIVE  IN  THESE  2  AREAS  OF  CIVICS 

66 

RDDIF 

-0 

170134 

AT  LEAST  SOME  DIFFICULTY  IN  READING  OR  MISSING 
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Table  C-27 


Estimated  Effects  for  Civics  Cross  sectional  Conditioning  Variables 
1988  Intercorrelation  Sample,  Grade  12/Age  17 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-0 

.911554 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER- F 

-0 

.155584 

SEX  (FEMALE) 

3 

ETHNIC-B 

-0 

.265358 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC-H 

-0 

.301223 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC-A 

-0 

.5&3830 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

STOC-H 

0 

.097614 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOC-X 

0 

.067059 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REG-SE 

0 

.091114 

REGION  (SOUTHEAST) 

9 

REG-C 

0 

.021948 

REGION  ^CENTRAL) 

10 

REG-W 

0 

.006807 

REGION  (WEST) 

11 

PAREDHS 

-0 

.134283 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

-0 

.069501 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDC 

-0 

.015859 

PARENTS  EDUCATION  (COLLEGE  GRADUATE) 

14 

PAREDM 

-0 

.136091 

PARENTS  EDUCATION  (MI.'^SING,  I  DON'T  KNOW) 

15 

ITEMS -3 

0 

.0724ri 

3  ITEMS  IN  THE  HOME 

16 

ITEMS  4 

0 

104143 

4  ITEMS  IN  THE  HOME 

17 

TVWATCH 

0 

140501 

NO.  HOURS  OF  TV  WATCHED  PER  DAY 

18 

TVWATCH2 

-0 

021632 

NO.  HOURS  SQUARED  OF  TV  WATCHED  PER  DAY 

'9 

KOMELANG 

0 

097374 

MINORITY  HOME  LANGUAGE  SPOKEN 

20 

HOMEWRK- 

0 

148167 

DON'T  HAVE  HOMEWORK 

21 

HOKEWRKO 

0 

185228 

DON'T  DO  HOMEWORK 

22 

HOMEWORK 

0 

066495 

AMOUNT  OF  HOMEWORK  DONE 

23 

PCTLUNCH 

0 

003626 

PERCENT        LUNCH  PROGRAM 

24 

NOLUNCH 

0 

021832 

DON'T  Hn.^  LUNCH  PROGRAM  OR  MISSING 

25 

PCTWHLOW 

-0 

145977 

PCT  WHITE  IN  SCHOOL  0-49 

26 

PCTWHMED 

-0 

011403 

PCT  WHITE  IN  SCHOOL  50-79 

27 

-MA:<MG 

-0 

038936 

AT  MODAL  AGE;  BELOW  MODAL  GRADE 

28 

-MA:-MG 

0 

138353 

AT  MODAL  AGE;  AT  MODAL  GRADE 

29 

>m.:~m 

0. 

098470 

ABOVE  MODAL  AGE;  AT  MODAL  GRADE 

30 

SCH-PRIV 

0. 

081950 

SCHOOL  OTHER  THAN  PUBLIC 

31 

HOMEHELP 

-0. 

183083 

USUALLY  GET  HELP  AT  HOME  WITH  HOMEWORK 

32 

HOMEPARS 

0. 

071978 

FATHER  AND  MOTHER  BOTH  AT  HOME 

33 

HOMEMOM 

0. 

081156 

MOTHER  LIVES  AT  HOME 

34 

MOMWORK 

0. 

026910 

MOTHER  WORKS  OUTSIDE  OF  HOME 

35 

RD05+PP 

0. 

145215 

READ  6+  PAGES  PER  DAY  FOR  SCHOOL 

36 

RDll+PP 

-0. 

066205 

READ  11+  PAGES  PER  DAY  FOR  SCHOOL 

37 

NOMISS 

0. 

040696 

DID  NOT  MISS  MUCH  SCHOOL  LAST  MONTH 

38 

G»ADES 

0. 

234408 

GRADES  IN  SCHOOL 

39 

COLLPREP 

0. 

118036 

COLLEGE  VREP.  PROGRAM 

40 

VOCTECH 

-0. 

064741 

VOCATIONAL/TECH.  PROGRAM 

41 

PLAN2YR 

0. 

059541 

PLAN  TO  GO  TO  2 -YEAR  COI^EGE 

(continued) 
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Table  C-27  (continued) 

Estimated  Efiects  for  Civics  Cross -sectional  Conditioning  Variables 
1988  Intercorrelation  Sample,  Grade  12/Age  17 


Estimated 


Variable 

Effect 

Description 

42 

PLAN4YR 

0. 

165027 

PLAN  TO  GO  TO  4 -YEAR  COLLEGE 

43 

WORKHRS 

-0. 

002365 

NO.  HOURS  OF  OUTSIDE  WORK 

44 

ENG-AP 

0. 

147936 

TAKING  A. P. /COLL.  PREP.  ENGLISH  CLASS 

45 

ENG-REM 

-0 

085238 

TAKING  REMEDIAL  ENGLISH  CLASS 

46 

STCV-INC 

-0 

292528 

STUDIED  CIVICS  -  IIICOMPLETE  SET  OF  ANSWERS 

47 

STCV9 

0 

012856 

STUDIED  CIVICS  IN  9TH  GRADE 

48 

STCVIO 

-0 

268831 

STUDIED  CIVICS  IN  lOTH  GRADE 

49 

STCVll 

-0 

083368 

STUDIED  CIVICS  IN  IITH  GRADE 

50 

STCV12 

0 

102932 

STUDIED  CIVICS  IN  12TH  GRADE 

51 

GOVT 

0 

.064298 

HOW  MANY  YEARS  OF  GOV'T  STUDY  HAVE  YOU  HAD 

52 

AP 

-0 

.070794 

ADVANCED  PLACEMENT  YES 

53 

ST-ALOT 

0 

.018307 

NUMBER  OF  SUBJECTS  STUDIED  A  LOT  (OUT  OF  10) 

54 

ST-AV 

-0 

.000833 

AVERAGE  LEVEL  OF  STUDY  FOR  THESE  TEN 

ERIC 
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Table  C-28 


Estimated  Effects  for  Civics  Trend  Conditioning  Variables 
1988  Trend  Sample,  Age  13 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-1 

833217 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENt^-F 

-0 

274014 

SEX  (FEMALE) 

3 

ETHNIC-B 

-0 

245560 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC-H 

-0 

193642 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC-0 

0 

022923 

OBSERVED  ETHNICITY  (OTHER) 

6 

STOC-H 

0 

251747 

SIZE  Ai^D  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOC-:: 

0 

228720 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REG-SE 

0 

024840 

REGION  (SOUTHEAST) 

9 

REG-C 

0 

065093 

REGION  (CENTRAL) 

10 

REG-W 

-0 

209815 

REGION  (WEST) 

11 

PAREDHS 

-0 

016727 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

0 

163282 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDC 

0 

287076 

PARENTS  EDUCATION  (COLLEGE  GRADUATE) 

14 

PAREDM 

-0 

279247 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

ITEMS -3 

0 

093471 

3  ITEMS  IN  THE  HOME 

16 

ITEMS -4 

0 

197622 

4  ITEMS  IN  THE  HOME 

17 

TVWATCH 

0 

081961 

NO.  HOURS  OF  TV  WATCHED  PER  DAY 

18 

TVWATCH2 

-0 

016805 

NO.  HOURS  SQUARED  OF  TV  WATCHED  PER  DAY 

19 

HOMELANG 

-0 

054940 

MINORITY  HOME  LANGUAGE  SPOKEN 

20 

HOMEWRT- 

-0 

290030 

DON'T  HAVE  HOMEWORK 

21 

HOMEWRkJ 

-0. 

013951 

DON'T  DO  HOMEWORK 

22 

HOMEWORK 

0. 

049850 

AMOUNT  OF  HOMEWORK  DONE 

23 

PCTWHLOW 

•0. 

192148 

PCT  WHITE  IN  SCHOOL  0-49 

24 

PCTWHMED 

0. 

157684 

PCT  WHITE  IN  SCHOOL  50-79 

25 

-  MODALG 

0. 

575586 

GRADE  IN  SCHOOL  IS  MODAL  GRADF 

26 

>  MODALG 

1, 

748053 

GRADE  IN  SCHOOL  IS  GREATER  THAN  MCDAL  GRADE 

27 

SCH-PRIV 

0. 

213468 

SCHOOL  OTHER  THAN  PUBLIC 

28 

GRADES 

0. 

426167 

GRADES  IN  SCHOOL  (4 -POINT  SCALE) 
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Table  C-29 


Estimated  Effects  for  Civics  Trend  Conditioning  Varf.ables 
1982  Trend  Sample,  Age  13 


Estimated 


Variable 

Effect 

Tescriptioa 

1 

OVERALI. 

-1. 

264436 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER-F 

-0. 

321234 

SEX  (FEMALE) 

3 

ETHNIC-B 

-0. 

533501 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC-H 

-0. 

215298 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC -0 

-0, 

108223 

OBSERVED  ETHNICITY  (OTHER) 

6 

STOC-H 

0. 

247733 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOC-X 

-0. 

027729 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REG-SE 

065470 

REGION  (SOUTHEAST) 

9 

REG-C 

-0, 

012846 

REGION  (CENTRAL) 

10 

REG-W 

-0, 

045416 

REGION  (WEST) 

11 

PAREDHS 

0 

225191 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

0 

409932 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDC 

0 

466953 

PARENTS  EDUCATION  (COLLEGE  GRADUATE) 

14 

PAREDM 

-0 

.052883 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

ITEMS -3 

0 

138441 

3  ITEMS  IN  THE  HOME 

16 

ITEMS -4 

0 

.274918 

4  ITEMS  IN  THE  HOME 

17 

TWATCH 

0 

.004248 

NO.  HOURS  OF  TV  WATCHED  PER  DAY 

18 

TWATCH2 

-0 

.002819 

NO.  HOURS  SQUARED  OF  TV  WATCHED  PER  DAY 

19 

HOMELANG 

0 

.097368 

MINORITY  HOME  LANGUAGE  SPOKEN 

20 

HOMEWRK- 

-0 

.311352 

DON'T  HAVE  HOMEWORK 

21 

HOMEWRKO 

-0 

.207615 

DON'T  DO  HOMEWORK 

22 

HOMEWORK 

0 

.005725 

AMOUNT  OF  HOMEWORK  DONE 

23 

PCTWHLOW 

-0 

.299501 

PCT  WHITE  IN  SCHOOL  0-49 

24 

PCTWHMED 

-0 

.036163 

PCT  WHITE  IN  SCHOOL  SO- 79 

25 

-  MODALG 

0 

.463258 

GRADE  IN  SCHOOL  IS  MCJAL  GRADE 

26 

>  MODALG 

1 

.302472 

GRADE  IN  SCHOOL  IS  GREATER  THAN  MODAL  GRADE 

27 

SCH-PRIV 

0 

.183278 

SCHOOL  OTHER  THAN  PUBLIC 

28 

GRADES 

0 

.339169 

GRADES  IN  SCHOOL  (4-POINT  SCALE) 
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T&ble  C-30 


Estimated  Effects  for  Civics  Trend  Conditionimg  Varieblei* 
1976  Trend  Sample,  Age  13 


Estimated 


Variable 

Effect 

DescriDtlon 

1 

OVERALL 

-0.865586 

OVERALL  CONSTANT  '1'  FOR  EVERYONt 

2 

G^ER-F 

-0.180073 

SEX  (FEMALE) 

3 

ETHNIC -B 

-0.612380 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC -H 

-0.553508 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC -0 

-0.394742 

OBSERVED  ETHNICITY  (OTHER) 

6 

STOC-H 

0.426680 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOC-X 

0.019128 

SIZE  AND  TYPE  OF  COMMl'NITY  (NOT  HIGH  OR  LOW) 

8 

REG-SE 

0.010113 

REGION  (SOUTHEAST) 

9 

REG-C 

0.139607 

REGION  (CENTRAL) 

10 

REG-W 

-0.032758 

REGION  (WEST) 

11 

PAREDHS 

0.235653 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

0.560744 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDM 

-0.003951 

PARENTS  EDUCATION  (MISSING,   i  DON'T  KNOW) 

14 

ITEMS -3 

0.105100 

3  ITEMS  IN  THE  HOME 

15 

ITEMS -4 

0.413272 

4  ITEMS  IN  THE  HOME 

16 

PCTWHLOW 

-0.040969 

PCT  WHITE  IN  SCHOOL  0-49 

17 

PCTWHMED 

0.023708 

PCT  WHITE  IN  SCHOOL  50-79 

18 

-  MODALG 

0.584018 

GRADE  IN  SCHOOL  IS  MODAL  GRADE 

19 

>  MODALG 

0.918076 

GRADE  IN  SCHOOL  IS  GREATER  THAN  MODAL  GRADE 
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Table  C-31 


Estimated  Effects  for  Civics  Trend    onditioning  Variables 
1988  Trend  Sample,  Age  17 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

2. 

537850 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER- F 

-0. 

354324 

SEX  (FEMALE) 

3 

ETHNIC -B 

-0. 

371432 

OBSERVED  ET''  ICITY  (BLACK) 

4 

ETHNIC-H 

-0. 

072776 

OBSERVED  ETl    ICITY  (HISPANIC) 

5 

ETHNIC -0 

-0. 

156620 

OBSERVED  ETHNICITY  (OTHER) 

6 

STOC-H 

0. 

217207 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOC-X 

0. 

187236 

SIZE  AND  T^TE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REG-SE 

0. 

0836A1 

REGION  (SOUTHEAST) 

9 

REG-C 

-0. 

002988 

REGION  (CENTRAL) 

10 

REG-W 

-0. 

226426 

REGION  (WEST) 

11 

PAREDHS 

0. 

092559 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

0 

315752 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDC 

0 

377825 

PARENTS  EDr    .TION  (COLLEGE  GRADUATE) 

14 

PAREDM 

-0 

029314 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

ITEMS- 3 

0 

093015 

3  ITEMS  IN  THE  HOME 

16 

ITEMS -4 

0 

108895 

4  ITEMS  IN  THE  HOME 

17 

TVWATCH 

J 

015740 

NO.  HOURS  OF  TV  WATCHED  PER  DAY 

18 

TVWATCH2 

-0 

004894 

NO.  HOURS  SQUARED  OF  TV  WATCHED  PcR  DAY 

19 

HOMELANG 

-0 

.103856 

MINORITY  HOME  LANGUAGE  SPOKEN 

20 

HOMEMRK- 

-0 

.189856 

DON'T  HAVE  HOMEWORK 

HOMEMRKO 

0 

.229131 

DON'T  DO  HOMEWORK 

22 

HOMEWORK 

-0 

.094754 

AMOUNT  OF  HOMEWORK  DONE 

23 

PCTWHLOW 

0 

.035583 

PCT  WHITE  IN  SCHOOL  0-49 

24 

PCTWHMED 

-0 

,013220 

i.-'CT  WHITE  IN  SCHOOL  50-79 

25 

-  MODALG 

0 

,406556 

GRADE  IN  SCHOOL  IS  MODAL  GRADE 

26 

>  MODALG 

0 

,550452 

GRADE  IN  SCHOOL  iS  GREATER  THAN  MODAL  GRADE 

27 

SCH-PRIV 

-0 

^  f\  ^  f\  f\ 
,002692 

SCHOOL  OTHER  THAN  PUBLIC 

28 

GRADES 

0 

.407650 

GRADES  IN  SCHOOL  (4 -POINT  SCALE) 

29 

HSPGM-C 

0 

.355545 

HIGH  SCH  PROGRAM  COLLEGE  PREP 

30 

HSPGM-V 

-0 

.148816 

HIGH  SCH  PROGRAM  VOCAT'L  OH  OTHER  NONGENERAL 

31 

ISS-YES 

0 

.150710 

DISCUSS  ISSUES  IN  COURSES 

32 

ISS-SOMX 

0 

.141650 

DISCUSS  ISSUES  IN  COURSES  SOMEWHAT  OR  MISSING 

33 

POL-YES 

0 

.394967 

DISCUSS  POLITICS  IN  COURSES 

34 

POL-SOMX 

0 

.259171 

DISCUSS  POL.  11.  COURSES  SOMEWHAT  OR  MISSING 

35 

NONEW-Y 

0 

.017690 

NOTHING  NEW  DISCUSSED  IN  COURSES  -  TRUE 

36 

NONEW-SX 

-0 

.005086 

NOTH.  NEW  DISC.  IN  CRSES  -  SOMEWHAT  OR  MISSING 

37 

KNOW-YES 

-0 

.015209 

KNOWLEDGE  TO  PARTICIPATE  -  TRUE 

38 

KNOW-SX 

.0 

.009888 

KNOWLEDGE  TO  PARTICIPATE  -  SOMEWHAT  OR  MISSING 

39 

NATL-D 

0 

.134013 

NATIONAL  DISCUSSIONS  ALMOST  EVERY  DAY 

40 

NATL-C 

-0 

.03676: 

NAT'L  DISCUSSIONS  1-2  TIMES  A  WEEK  OR  MISSING 

41 

NATL-B 

-0 

.050476 

NATIONAL  DISCUSSIONS  3-4  TIMES  A  MONTH 

(continued) 
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Table  C-31  (continued) 


Estimated  Effects  for  Civics  Trend  Conditioning  Variables 
1988  Trend  Sample,  Age  17 


Variable 

42  INTL-D 

43  INTL-C 

44  INTL-B 


Estimated 
Effect 

0.221026 
0.152830 
0.181321 


INTEilNATIONAL  DISCUSSIONS  ALMOST  EVERY  DAY 
INT'L  DISCUSSIONS  1-2  TIMES  A  WEEK  OR  MISSING 
INTERNATIONAL  DISCUSSIONS  3-4  TIMES  A  MONTH 


RIC 
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Table  C-32 


Estimated  Effects  for  Civics  Tren^,  Conditio  ling  Variables 
1982  Trend  Sample,  Age  17 


Estimated 


Variable 

Effect 

Description 

1 

OVERALL 

-1 

361751 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER- F 

-0 

425031 

SEX  (FEMALE) 

3 

ETHNIC -B 

-0 

470033 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC -H 

-0 

296556 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC-0 

-0 

421034 

OBSERVED  ETHNICITY  (OTHER) 

6 

STOC-H 

0 

446878 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOC-X 

0 

203226 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOU) 

w  X            CM*  ly     X  X  L          V/&      V>\/i  11  Ivll  XXX      ^Ll\/X     1IXV71I     \/l\     l^V^  Tl  f 

8 

REG-SE 

0 

.034580 

REGION  (SOUTHEAST) 

i\ i>V7 X \J  11     ^w*/wxic w  X  y 

9 

REG-C 

0 

.050915 

REGION  (CENTRAL) 

i\i>v7x\/ii     I  vi>ii  & i\x»ijy 

10 

REG-W 

-0 

013089 

REGION  (WEST) 

i\  i^v7  X  V 11     \  "  M  w  X  y 

11 

PAREDHS 

0 

.116176 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

0 

267456 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDC 

0 

2837G5 

PARENTS  EDUCATION  (COLLEGE  GRADUATE) 

14 

PAREDM 

-0 

334944 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

ITEMS -3 

0 

165990 

3  ITEMS  IN  THE  HOME 

16 

ITEMS-4 

0 

275573 

4  ITEMS  IN  THE  HOME 

17 

TVWATCH 

0 

018188 

NO.  HOURS  OF  TV  WATCHED  PER  DAY 

18 

TVWATCH2 

-0 

.008675 

NO.  HOURS  SQUARED  OF  TV  WATCHED  PER  DAY 

19 

HOMELANG 

-0 

.135970 

MINORITY  HOME  LANGUAGE  SPOKEN 

20 

HOMEWRK- 

0 

.302385 

DON'T  HAVE  HOMEWORK 

21 

HOMEWRKO 

-0 

.090445 

DON'T  DO  HOMEWORK 

22 

HOMEWORK 

0 

004198 

AMOUNT  OF  HOMEWORK  DONE 

23 

PCTWHLOW 

-0 

090147 

PCT  WHITE  IN  SCHOOL  0-49 

24 

PCTWHMED 

-0 

044514 

PCT  WHITE  IN  SCHOOL  50-79 

25 

-  MODALG 

0 

.497047 

GRADE  IN  SCHOOL  IS  MODAL  GRADE 

26 

>  MODALG 

0 

662263 

GRADE  IN  SCHOOL  lo  GREATER  THAN  MODAL  GRADE 

27 

SCH-PRIV 

-0 

.  140064 

SCHOOL  OTHER  THAN  PUBLIC 

28 

GRADES 

0 

283818 

GRADES  IN  SCHOOL  (4- POINT  SCALE) 

29 

HSPGM-C 

0 

.403098 

HIGH  SCH  PROGRAM  COLLEGE  PREP 

30 

HSPGM-V 

-0 

009976 

HIGH  SCH  PROGRAM  VOCAT'L  OR  OTHER  NONGENERAL 
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Table  C-33 


Estimated  Effects  fox  Civics  Trend  Conditioning  Variables 
1976  Trend  Sample,  Age  17 


Estimated 


Variable 

£f£s£t 

Description 

OVERALL 

-1.564018 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

GENDER- F 

-0 

.439323 

SEX  (FEMALE) 

3 

ETHNIC-B 

-0 

.375957 

OBSERVED  ETHNICITY  (BUCK) 

4 

ETHNIC-H 

-0 

.411097 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

fc.THNIC-0 

-0 

.351636 

OBSERVED  ETHNICITY  (OTHER) 

6 

STOC-H 

0 

193189 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOC-X 

0 

.033124 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REG-SE 

-0 

062455 

REGION  (SOUTHEAST) 

9 

REG-C 

0 

059840 

REGION  (CENTRAL) 

10 

REG-W 

-0 

.024240 

REGION  (WEST) 

11 

PAREDHS 

-0 

005646 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PAREDHS+ 

0 

215860 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAREDM 

-0 

015286 

PARENTS  EDUCATION  {MISSIir\  I  DON'T  KNOW) 

14 

ITEMS -3 

0 

226880 

3  ITEMS  IN  THE  HOME 

15 

ITEMS-4 

0 

358708 

4  ITEMS  IN  THE  HOME 

16 

TVWATCH 

0 

003013 

NO.  HOURS  OF  TV  WATCHED  PER  DAY 

17 

TVWATCH2 

-0 

004743 

NO.  HOURS  SQUARED  OF  TV  WATCHED  PER  DAY 

18 

HOMELANG 

-0 

087508 

MINORITY  HOME  LANGUAGE  SPOKEN 

19 

HOMEWRK- 

-0 

543340 

DON'T  HAVE  HOMEWORK 

20 

HOMEWRKO 

-0 

333045 

DON'T  DO  HOMEWORK 

21 

HOMEWORK 

0 

0^6222 

AMOUNT  OF  HOMEWORK  DONE 

22 

PCTWHLOW 

-0 

092577 

PCT  WHITE  IN  SCHOOL  0-49 

23 

PCTWHMED 

-0 

005133 

PCT  WHITE  IN  SCHOOL  50-79 

24 

-  MODALG 

0 

402627 

GRADE  IN  SCHOOL  IS  MODAL  GRADE 

25 

>  MODALG 

0 

624665 

GRADE  IN  SCHOOL  IS  GREATER  THAN  MODAL  GRADE 

26 

GRADES 

0. 

424681 

GRADES  IN  !>CHOOL  (4 -POINT  SCALE) 

27 

HSPGM-C 

0. 

270935 

HIGH  SCH  PROGRAM  COLLEGE  PREP 

28 

HSPGM-V 

201353 

HIGH  SCH  PROGRAM  VOCAT'L  OR  OTHER  NONGENERAL 

29 

ISS-YES 

-0. 

022408 

DISCUSS  ISSUES  IN  COURSES 

30 

ISS-SOMX 

0. 

010048 

DISCUSS  ISSUES  IN  COURSES  SOMEWHAT  OR  MISSING 

31 

POL-YES 

0. 

140908 

OISCUSS  POLITICS  IN  COURSES 

32 

POL-SOMX 

0. 

086860 

DISCUSS  POLITICS  IN  CRSES  SOMEWHAT  OR  MISSING 

33 

NOl'vW-Y 

0. 

021515 

NOTHING  NEW  DISCUSSED  IN  COURSES  -  TRUE 

34 

NONW-SX 

0. 

066948 

NOTH.  NEW  DISC.   IN  CRSES  -  SOMEWHAT  OR  MISSING 

35 

KNOW-YES 

-0. 

184577 

KNOWLEDGE  TO  PARTICIPATE  -  TRUE 

36 

KNOW-SX 

•0. 

027103 

KNOWLEDGE  TO  PARTICIPATE  -  SOMEWHAT  OR  MISSING 

37 

NATL-D 

0. 

?12025 

NATIONAL  DISCUSSIONS  ALMOST  EVERY  DAY 

38 

NATL-C 

0 

67067 

NAT'L  DISCUSSIONS  1-2  TIMES  A  WEEK  OR  MISSING 

39 

NATL-B 

0. 

105106 

NATIONAL  DISCUSSIONS  3-4  TIMES  A  MONTH 

40 

INTL-D 

0. 

054382 

INTERNATIONAL  DISCUSSIONS  ALMOST  EVERY  DAY 

41 

INTL-C 

0. 

032460 

■'NT'L  DISCUSSIONS  l-.l  TIMES  A  WEEK  OR  MISSING 

42 

INTL-B 

0. 

091534 

INTERNATIONAL  DISCUSSIONS  3-4  TIMES  A  MONTH 

ERIC 
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Table  C-34 


Estimated  Effects  for  U.S.  History  Cross-sectional  Conditioning  Variables 
1988  Focused-BIB  and  Intercorrelat? on  Samples,  Grade  4/Age  9 


Estimated 

Variable 

i 

OVERALL 

-1 

082049 

OVERALL  CONSTANT  ' 1 '   FO^  EVERYONE 

2 

GENDER2 

-0 

0<J2760 

SEX  fFE^fALE^ 

3 

ETHNIC 2 

tat  Jk  I  U«  X  W  ^ 

-0 

211443 

OBSERVED  ETHNICITY  TBUVrK^ 

4 

ETHNIC 3 

-0 

302839 

OBSERVED  FTHNiriTV  ^HT^PANTP^ 

\J  lJ*JL^S\y  L^U     UXILLIXOXXX      ^  41  X  «3X          X  o  ^ 

5 

ETHNIC4 

0 

084755 

OBSERVED  FTHNiriTY  TASIAN^ 

6 

oT0C2 

0 

256330 

<5T7F  ArJn  TYPF  OF  rOMMTTNTTY   ^KTf^H  MFTRO^ 

O  X  i^Lt    t\L^U     X  X  X  Xf    V/X     OV/lxnULi  XXI     ^  II X Oil    lib  X  C\V/  / 

7 

ST0C3 

0 

099060 

^T7F  AND  TYPF  OF  rOMMTTNTTY   ^NOT  HTfiH  OR  1  fiU^ 

8 

REGI0N2 

0 

060228 

REGION  fSOUTHEAST^ 

9 

REG ION 3 

0 

015842 

REGION  f  CE^'TRAT  ) 

CVX«V7  X  V/ll      \  ^       '  X 

10 

REGI0N4 

-0 

054248 

R!iGION  ^UFST^ 

11 

PARED? 

0 

070426 

PARFNT*;  FDITPATinN   ^HTfiH  '^PHnni  ^PA^^ 

12 

PARED 3 

0 

205145 

PARFNT^  FDIirATinN   ^PO^T  HlfiH  ^JTHnnT  ^ 

13 

PARI 

0 

296709 

PARENTS  EDUCATION  ^rOTTFGF  GRAD^ 

14 

PARL, 

0 

079094 

PARENTS  EDUCATION  ^MISSING     I  Don'T  KIJOW^ 

15 

H0MEITM3 

0 

086135 

ARTICLES  IN  HOME  fYES  TO  ''^ 

16 

H0MEITM4 

0 

.183931 

ARTICLES  IN  HOME  (YES  TO  4; 

17 

TV.  1 

0 

.164997 

TV  WATCHING 

18 

TV.  2 

-0 

.243077 

TV  WATCHING 

19 

HOMELNGl 

0 

001665 

OTHER  LANGUAGE  AT  HOME  ^ SOMETIMES  ALWAYS^ 

20 

HW-NO 

-0 

050667 

HOMEWORK  (NONE  ASSIGNED) 

21 

HW-YES 

-0 

021858 

HOMEWORK  (YES  -  SOME  AMOUNT) 

22 

HW-2345 

-0 

008786 

HOMEWORK  AMOUNT  (LINEAR) 

23 

ZLUNCH  1 

-0 

014315 

PERCENT  IN  PROGRAM 

24 

XLUNCH  2 

-0 

.158767 

BLANK 

25 

XWHITE  1 

-0 

.121124 

WHITE  MINORITY 

26 

XWHITE  2 

-0 

.039429 

INTEGPATED 

27 

AGE/GRDl 

-0 

.562740 

MODAL  AGE,  <  MODAL  GRADE 

28 

AGE/GRD2 

-0 

076542 

MODAL  AGE.  MODAL  GRADE  MISSING 

29 

AGE/GRD3 

0 

338196 

MODAL  AGE    >  MODAL  GRADE 

30 

AGE/GRD4 

-0 

.190299 

>  MODAL  AGE»  MODA;.  GRADE 

31 

SCH.TYPE 

0 

.113039 

PUBLIC  V.  PRIVATE  'CHOOLS 

32 

H/HOMEWK 

-0 

.106155 

HELP  WITH  HOMEWO?jC  ('iVERY  DAY,  TWICE  A  WEEK) 

33 

PRESCH 

0 

138178 

WENT  TO  PRESCHOOL 

34 

PAR/HOME 

0 

029196 

SINGLE/MULTIPLE  PARENT  (MULTIPLE) 

35 

MOTH/HOM 

0 

.125032 

MOTHER  AT  HOME  (YES) 

36 

MOTH/WRK 

-0 

018852 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

37 

GRWN  HOM 

0 

.062107 

GROWNUP  AT  HOME  RIGHT  AFTER  SCHOOL 

38 

PAGE/ROl 

0 

053224 

PAGES  A  DAY  READ     (6  /  O  UP) 

39 

PAGE/R02 

0 

047888 

PAGES  A  DAY  READ     (11  AND  UP) 

40 

HIS  TAKE 

-0 

015240 

HOW  OFTEN  DO  YOU  HAVE  A  SOCIAL  STUDIES  CLASS 

41 

JTUD.HIS 

0 

.068609 

STUDIED  THE  HISTORY  OF  OUR  COUNTRY 

(continued) 
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Table  C-34  (continued) 


Estimated  Effects  for  U.S.  History  Cross-sectional  Conditioning  Variables 
1988  Focused-BIB  and  Intercorrelation  Samples,  Grade  4/Age  9 


Estimated 


Variable 

Effect 

Description 

42 

STUD. EXP 

0 

.044026 

STUDIED  ABOUT  THE  E.ARLY  EXPLORERS 

43 

STUD.IND 

-0 

.002413 

STUDIED  ABOUT  THE  AMERICAN  INDIANS 

44 

STUD.PIL 

-0 

.001754 

STUDIED  ABOUT  THE  PILGRIMS 

45 

STUD.GW 

-0 

.045748 

STUDIED  ABOUT  GEORGE  WASHINGTON 

46 

STUD.PIO 

0 

.031049 

STUDIED  ABOUT  THE  PIONEERS 

47 

2TUD.SLA 

-0 

.011719 

STUDIED  ABOUT  SLAVERY 

48 

STUD.INV 

-0 

.029672 

STUDIED  ABOUT  PEOPLE  WH"  INVENTED  THINGS 
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Table  C-35 


Estimated  Effects  for  U.S.  History  Cross -se.^.tional  Conditioning  Variables 
1988  Focused-BIB  and  Intercorrelation  Jamplos,  Grade  8/Age  13 


Estimated 

V  dJL  J.CtLI  XC? 

Effect 

L/coCx  xp  Ci-UIl 

1 

-0 

.833818 

nUFPATT    rnN<!TANT    ' '    FOP  FVFPVnNF 

0 

-0 

.150065 

-0 

.175538 

nR^FPUFD  FTHNTPTTY  ^RTJVPlf^ 

A 

i!ti.rui  J 

-0 

.235940 

nRQFPUPn  FTHMTfTTV  ^HT^JPANTP^ 

s 

-0 

.047478 

nRQPPUPn  FTUMTPTTY  ^AQTAK^^ 
KJDO EtSxy  CtU    I!tlIuM  XUX  X  X    ^AD  X/lii  } 

f. 
\j 

0 

.086050 

^T7F  AWn  TYPF  OF  rnXfMTTNTTV   ^HTfiH  MFTPO^ 
Dx^c  £1x11/  xxxc  vjr   uufinurixxx   ^nxon  ncxixu^ 

7 

0 

.044790 

^T7F  AKH  TYP^7  OF  mMMlTNTTY   ^NOT  HTpu  np  Tnu^ 
oL£sC*  t\iiu  X  Xx  c*  vjr   uunnvrixxx   ^iiui  njk.un  ui\  lajyi  j 

» 

RFGT0N9 

-0 

.068347 

PFflTnN  r^OUTHFAST^ 

0 

0 

.008025 

PFfiinN   ^PFNTPAT  ^ 

in 

-0 

.025248 

1 1 

JLJL 

0 

.046675 

PAPFMTQ  FDnrATinfj  ^HinH  ^pHnnv  ^PA^^ 

1 9 

0 

.163076 

PAPFKTTQ   pnTTfATTnM   fVn<^T  HT14   ^PHnflT  ^ 

1  'X 

PAPPHA 

0 

.162114 

PAPFMT^   FniTPATTriN    ^mTTFPF  fiPAH^ 

i*+ 

PAPPn 

-0 

.040112 

PAPFMT^.   FIMTPATTriM    ^MT^^TNP      T   BHN'T  I^MnU^ 

0 

.073651 

0 

.139044 

17 

JL  / 

TV  1 

i.  V  .  JL 

0 

.063579 

TV  UATGHTNG 

TV  9 

-0 

.107249 

TV  UATGHTNG 

X  V     wn  X  oil  X  Li  V7 

19 

-0 

.01"^  ■'49 

OTHER  LANGUAGE  AT  HOME  f SOMETIMES  ATUAYS^ 

20 

HW-NO 

0 

.0'  54 

HOMEWORK  fNONE  ASSIGNED^ 

21 

HW-YES 

0 

.253431 

HOMEWORK  (YES  -  SOME  AMOUNT) 

22 

HW-2345 

0 

.023776 

HOMEUORK  AMOUNT  f LINEAR^ 

23 

-0 

.011917 

PFRGFNT  TN  PROGRaM 

X  A^IxOULl  X      XLl     X  lVV/V7lAfUA 

24 

tTUNCH  2 

-0 

.020277 

UT  AW 

25 

tUHTTF  1 

-0 

.024245 

I"!TTE  MTNORTTY 

V.M&XXAJ     L AX  11  V/AxX  X  X 

yrjHTTF  9 

0 

.007425 

TNTFGRATFD 

Xll  X  UV7CvrVX  ux/ 

27 

AGE/GRDl 

-0 

.387768 

MODAT  AGF    <  MODAT  GRADE 

LAV/iyflAj    £\vM  1      ^    LAV/X/nAj  V7A\X»iyx« 

9ft 

AfiF  /npn9 

-0 

.124035 

MODAT    AGF     MODAT    GRADF  MT<5<5TNG 

9Q 

Af2F  /nPH'i 

0 

.170160 

MODAT    ACF          MODAT  PPADF 

30 

AGE/GRD4 

-0 

.208910 

>  MODAL  AGE,  MODAL  GRADE 

31 

SCli.TYPE 

0 

.056116 

PUBLIC  V.  PRIVATE  SCHOOLS 

32 

HA  'MEVJK 

-0 

.142969 

HELP  WITH  HOMEWORK  (EVERY  DAY,  TWICE  A  WEEK) 

33 

PAR/HOME 

0 

.004462 

SINGuE/MULTIPLE  PARENT  (MULTIPLE) 

34 

MOTH/HOM 

0 

.075370 

MOTHER  AT  HOME  (YES) 

35 

MOTH/WRK 

-0 

.022306 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

36 

PAGE/RDl 

0 

.078121 

PAGES  A  DAY  READ     (6  AND  UP) 

37 

FAGE/RD2 

~0 

.007233 

PAGES  A  DAY  READ     (11  AND  UP) 

38 

GRAD  HS 

0 

.262266 

DO  YOU  EXPECT  TO  GRADUATE  HIGH  SCHOOL 

39 

DAYS/KIS 

0 

.054914 

DAYS  OF  SCHOOL  MISSED  LAST  MONTH  (NONE) 

40 

GRADES 

0 

.221464 

GRADES  IN  SCHOOL 

41 

HIS  TAKE 

-0 

.003141 

HISTORY  COURSES  TAKEN  IN  GRADES  5-8 

(continued) 
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Table  C-35  (continued) 

Estimated  Effects  for  U.S.  History  Cross -sectional  Coi.  'tioning  Variables 
1988  Focused-BIB  and  Intercorrelation  Samples,  Grade  8/Agc  13 


42 
43 
44 
45 


Variable 

EARLY  H 
POST  C  H 
TEACH  H 
TEXTBOOK 


Estimated 
Effect 

0.338573 
-0.174067 
-0.1507:9 
■0.077157 


Description 

STUDIED  EARLY  US  HISTORY 
STUDIED  POST  CIVIL  WAR 

THINGS  TEACHER  ASKS  FOR  SOCIAL  STUDIES  CLASS 
DIFFICULTY  READING  US  HISTORY  TEXTBOOK 
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Table  C-36 


Estimated  Effects  for  U.S.  History  Cross-sectional  Conditioning  Variables 
1988  Focused-BIL  and  Intercorrelation  Samples,  Grade  12/Age  17 


Estimated 

Variable 

Effect 

Description 

1 

OV  'ALL 

-0. 

535861 

OVERALL  CONSTANT  '1'  FOR  EVERVonE 

2 

GENDER2 

-0. 

171101 

SEX  (FEMALE) 

3 

ETHNIC2 

-0. 

277811 

OBSERV  0  ETHNICITY  (BLACK) 

4 

ETHNIC3 

-0. 

249072 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

-0. 

145208 

OBSERVED  ETHNICITY  (ASIAN) 

6 

ST0C2 

0. 

131126 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0 

086993 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

-0 

030869 

REGION  (SOUTHEAST) 

9 

REGI0N3 

0. 

029813 

REGION  (CENTRAL) 

10 

REGI0N4 

0. 

015950 

REGION  (WEST) 

11 

PARED2 

0 

048557 

PARENTS  EDUCx.iiON  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0 

125807 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0, 

147278 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PALBD_ 

0, 

093690 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

KC  'EITM3 

0 

058395 

ARTICLES  i::  HOME  (YES  TO  3) 

16 

H0IiEITM4 

0 

1081-'5 

ARTICLES  IN  HOME  (YES  TO  4) 

17 

TV.  1 

0 

048061 

TV  WATCHING 

18 

TV.  2 

-0 

006601 

TV  WATCHING 

19 

HOMELNGl 

-0 

027066 

OTHER  LANGUAGE  AT  HOME  (SOMIiTIMES,  ALWAYS) 

20 

HW-NO 

0 

311514 

HOMEWORK  (NONE  ASSIGNED) 

21 

HW-YES 

0 

438620 

HOMEWORK  (YES  -  SOME  AMOUNT) 

22 

HW-2345 

-0 

013176 

HOMEWORK  AMOUNT  (LINEAR) 

23 

%LONCH  1 

-0 

.C00294 

PERCENT  IN  PROGRAM 

24 

XLUNCH  2 

0 

.006432 

BLANK 

2'j 

XWHITE  1 

-0 

.057512 

WHITE  MINORITY 

26 

ZWHITE  2 

-0.022398 

INTEGRATED 

27 

AGE/GRDl 

-0 

.191419 

MODAL  AGE,  <  MODAL  GRADE 

28 

\GE/GRD2 

-0 

.067168 

MODAL  AGE,  MODAL  GRADE,  MISSING 

29 

AGE/GRD4 

-0 

.122420 

>  MODAL  AGE,  MODAL  GRADE 

30 

SCH.TYPE 

-0 

.062657 

PUBLIC  V.  PRIVATE  SCHOOLS 

31 

H/HOMEWK 

-0 

.190038 

KELP  WITH  HOMEWORK  (E^/ERY  DAY,  TWICE  A  WEEK) 

32 

PAR/HOME 

0 

.005518 

SINGLE/MULTIPLE  PARENT  (MULTIPLE) 

33 

MOTH/HOM 

0 

.165298 

MOTHER  AT  HOME  (YES) 

34 

MOTH/WRK 

-0 

.008933 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

35 

PAGE/RDl 

0 

.069346 

PAGES  A  DAY  READ     (6  AND  UP) 

36 

PA0E/RD2 

0 

.059610 

PAGES  A  DAY  READ    (11  AND  UP) 

37 

DAYS/MIS 

0 

.037586 

DAYS  OF  SCHOOL  MISSED  LAST  MONTH  (NONE) 

38 

GRADES 

0 

.148587 

GRADES  IN  SCHOOL 

39 

HS  PROGl 

0 

.157131 

COLLEGE 

40 

HS  PR0G2 

-0 

.049570 

VOCATIONAL,  TECHNICAL 

41 

POST  S  1 

0.041032 

TWO-YEAR  COLLEGE 

(continued) 
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Table  C-36  (continued) 

Estimated  Effects  for  U.S.  History  Cioss- sectional  Conditioning  Variables 
1988  Focused-BIB  and  Intercorrelation  Samples,  Grade  12/Age  17 


Estimated 

Variable 

Effect 

Description 

42 

POST  S  2 

U 

FOUR-YEAR  COLLEGE 

43 

HRS  WORK 

-0 

001431 

HOURS  OF  OUTSIDE  WORK 

44 

ENG  CL  1 

0 

128965 

TYPE  OF  ENGLISH  CLASS  ADVANCED/COLLEGE 

45 

ENG  CL  2 

-0 

196378 

TYPE  OF  ENGLISH  CLASS  REMEDIAL 

46 

HIS  TAKE 

-0 

082710 

HISTORY  COURSES  TAKEN  IN  GRADES  9-12 

47 

mZ  COUR 

-0 

000022 

US  HISTORY  COURSE  C0MPLE7E 

48 

AP  HIST 

0 

.034608 

TAKEN  ADVANCED  PLACEMENT  AMERICAN  HISTORY 

49 

AH  HIST 

0 

195041 

STUDIED  AMERICAN  HISTORY 

50 

WORLD  H 

-0 

.088624 

STUDIED  WORLD  HISTORY 

51 

M:rN/"WOM 

-0 

.071197 

STUDIED  HISTORY  OF  MINORITIES  P^h  WOMEN 

52 

LIKE  H 

0 

.112180 

DO  YOU  LIKE  STUDYING  US  HISTORY 

53 

HAD  HIST 

0 

.045186 

EVER  HAD  A  US  HISTORY  CLASS 

54 

HOMEWK  H 

0 

.023091 

TIME  SPENT  EACH  WEEK  ON  US  HIST.  lOMEWORK 

55 

GRADES  H 

0 

.071975 

GRADES  YOU  HAVE  GOTTEN  ON  US  HISTORY 

56 

TEACH/H 

-0 

. 104300 

THINGS  TEACHER  ASKS  FOR  US  HISTORY  CLASS 

57 

TEXTBOOK 

-0 

.080254 

D  .FFICULTY  READING  US  HISTORY  TEXTBOOK 
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Table  C-37 


Estimated  Effects  for  Geography  Cross-sectional  Conditioning  Variables 
1988  Focused-BIB  and  Intercorrelation  Samples,  Grade  12/Age  17 


Estimated 

Variable 

Effect 

Description 

1 

OVERALL 

-0. 

332830 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

-0. 

420687 

SEX  (FEMALE) 

i 

ETHNIC  2 

-0. 

685879 

OBSERVED  ETHNICITY  (BLACK) 

ETHNIC 3 

-0. 

418^85 

OBSERVED  ETHNICITY  (HISPANIC) 

c 

ETHNIC4 

-0. 

184878 

OBSERVED  ETHNICITY  (ASIAN) 

6 

ST0C2 

0. 

269361 

SIZE  AND  TYPE  OF  COMMUNITY  (.HIGH  METRO) 

7 

S10C3 

0. 

169736 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

o 
0 

REGI0N2 

-0 

011230 

O  T^/^X  AM      /  C  AT  I'l'ITT^  A  OT'  \ 

REGION  (SOUTHEAST) 

9 

REGI0N3 

0 

024789 

REGION  (CENTRAL) 

REGI0N4 

0 

047082 

REGION  (WEST) 

11 

PARED2 

0 

113712 

PiVRENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED 3 

0 

279869 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

1  o 
ij 

PARED4 

0 

287398 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

-0 

158917 

A  n  T^MTtf     T^T\T  IO  A  TtX  AVT      /  V#X  O  O  X  XTO         X      T\AXT  #  »T»  ITXTATTX 

PARENTS  EDUCATION  (MISSING,  I  DON  T  KNOW) 

ID 

HUMbiTrlj 

u 

:.17G76 

AOTX/^T  T7C     TKT    TJAUT?     /VT7C    TA     T  \ 

AKlluLhb  IN  HOMH#  (,ibo  iU 

io 

IJ  ALff  T?  X  'PXl  A 

nUHhiln4 

0 

196076 

ADTX/^T  r*C»     TKT    TJAviT?     /VT?C    TA     /.  \ 

AKlluLLo  IN  HOME  (ibo  iU  4) 

i/ 

TXT  1 

0 

.042435 

i V  WAiOniNb 

lo 

TV.  Z 

-0 

.087285 

TV  WATCHING 

HOnEL^Gl 

0 

.021254 

OTHER  LANbUAbb  Ai  HOME  (bUHhllMho,  AlWAYb^ 

on 

UTJ  MO 

-G 

.659565 

O  1 

Zi 

UTT  VCC 

nW- ihb 

■0 

.460317 

HOMEWORK  (ibb  -  bUMh  AMUUNl) 

zz 

UTJ   O  Q  A  8^ 

aW-Zj'flJ 

0 

.024182 

llAlJfT?TTADV    AMAlTKTT  /TTKII7Al>\ 

0^5 

z^ 

0 

.000708 

"DTrDOTrKTT    TKI  "DDAr'OAM 

r EiKLp EiN i  IN  r KUuKArl 

z«+ 

VTTTMOU  O 

XLUr<Un  Z 

-0 

.089351 

OTA  XTl/ 

BLANK 

ZD 

ZWHiih  i 

-0 

.107201 

Wnlih  nxNUKiii 

0£ 

Zo 

VTTtIX»T»T7  O 

ZWHITE  z 

-0 

.068820 

INlhCyKAlhU 

O  "7 
Z  / 

-0 

.415062 

MODAL  AGE,  <  MODAL  GRADE 

Zo 

AGE/GRDz 

-0 

.251628 

\Mf\T\  AT     A  y**"^      Il/iM\  a  t          a  ^\T7      xf  t  cot  XT^' 

MODAL  AG^-.  MODAL  GRADE,  MISSING 

29 

AGE/GRD4 

-0 

.282149 

>  MODAL  AGE,  XODAL  GRADE 

30 

SCH.TYPE 

-0 

.138944 

PUBLIC  V.  PRIVATE  SCHOOLS 

31 

H/HOMEWK 

-0 

.245424 

HELP  WITH  HOMEWORK  (EVERY  DAY,  TWICE  A  WEEK) 

32 

PAR/HOME 

-0 

.014887 

SIITGLE/M-JLTIPLE  PARENT  (MULTIPLE) 

33 

MOTH/HOM 

0 

.119619 

MOTHER  AT  HOME  (YES) 

34 

MOTH/WRK 

-0 

.027926 

MOTHER  WORK  OUTSIDE  HOME  (YES) 

35 

PAGE/RDl 

0 

.031073 

PAGES  A  DAY  READ     (6  AND  'JJ) 

?6 

PAGE/RD2 

0 

.091392 

PAGES  A  DAY  READ     (11  AND  UP) 

37 

DAYS/MIS 

0 

.065005 

DAYS  OF  SCHOOL  MISSED  LAST  MONTH  (NONE) 

38 

GRADES 

0 

.213982 

GRADES  IN  SCHOOL 

39 

HS  PROGl 

0 

.212703 

COLLF.GE 

40 

PR0G2 

0 

.057015 

VOCATIONAL,  TECHNICAI, 

41 

POST  S  1 

0 

.042615 

TWO-YEAR  COLLEGE 

(continued) 
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Table  C-37  (continued) 


Estimated  Effects  for  Geography  Cross -sectional  Conditioning  Variables 
1988  Focused-BXB  and  Intercorrelation  Samples    ^rade  12/Age  17 


Estimated 


Variable 

Effect 

Description 

42 

POST  S  2 

0 

.232319 

FOUR-YEAR  COLLEGE 

43 

HRS  WORK 

-0 

.002'*  ^ 

HOURS  OF  OUTSIDE  WORK 

44 

ENG  CL  1 

0 

.1141U/ 

TYPE  OF  ENGLISH  CLASS  ADVANCED/COLLEGE 

45 

ENG  CL  2 

-0 

. 3032^8 

TYPE  OF  ENGLISH  CLASS  REMEDIAL 

46 

GEO  TAKE 

0 

.004362 

GEOGRAPHY  COURSES  TAKEN  IN  GRADES  9-12 

47 

W  GEO 

-0 

.073306 

WORLD  GEOGRAPHY  COURSE  COMPLETE 

48 

W  HIS/G 

-0 

.105087 

WORLD  HISTORY/GEO  COURSE  COMPLETE 

49 

STAT/REG 

0 

105754 

STATE/REGIONAL  COURSE  COMPLETE 

50 

US  GEO 

-0 

140865 

UNITED  STAES  COURSE  COMPLETE 

51 

US  H/G 

0 

180866 

UNITED  SATES  HIST/GEO  COURSE  COMPLETE 

52 

PHY/GEO 

-0 

082555 

PHYSICAL  GEG/EARTH  SCIENCE  COURSE  COMPLETE 

53 

ECON/P 

-0 

048159 

ECONOMIC  AND  POLITICAL  COURSE  COMPLETE 

54 

HUM/CULT 

0 

190482 

HUMAN  AND  CULTURAL  COURSE  COMPLETE 

55 

URBAN  G 

-0 

144868 

URBAN  COURSE  COMPLETE 

56 

PHYSICAL 

0 

083896 

SUM  OF  PHYSICAL  GEO  TOPICS  STUDIED 

57 

-SOCIAL 

-0 

089169 

SUM  OF  SOCIAL  GEO  TOPICS  STUDIED 
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Table  C-38 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 
1988  Bridge  to  1986,  Age  9 


Estimated 

Variable 

Effect 

Description 

1 

OVERALL 

-0 

279547 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

-0 

047747 

GENDER  (FEMALE) 

3 

ETHNIC2 

-0 

706632 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC  3 

0 

209298 

OiSSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

0 

762678 

OBSERVED  ETHNICITY  (ASIAN) 

6 

ST0C3 

0 

186615 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

STOCl 

0 

087756 

SIZE  AND  ITPE  OF  COMMUNITY  (NOT  HI6cN0T  LO) 

8 

REGI0N2 

0, 

007280 

REGION  (SOUTHEAST) 

9 

REGIONS 

0 

123942 

REGION  (CENTRAL) 

10 

REGT0N4 

-0 

035032 

REGION  (WEST) 

11 

PARED2 

0 

251057 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0 

223869 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

454556 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED 

0 

136615 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

<MODAL  GRADE 

-0 

728308 

MODAL  GRADE   XESS  THAN  MODAL  GPJ^DE) 

16 

>MODAL  GRADE 

0 

631198 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE; 

17 

ITEMS2 

0 

239816 

ITEMS  IN  THE  HOME  (YES  TO  3) 

18 

ITEMS3 

0 

367498 

ITEMS  IN  THE  HOME  (YES  TO  ALL  4) 

19 

E2  X  ^EX 

0 

087308 

ETHNICITY 

BY  GEND^     (BLACK,  FEMALE) 

20 

E3  X  SEX 

-0 

066049 

ETH^'ICITY 

BY  GENDh     (HISPANIC,  FEMALE) 

21 

E4  X  SEX 

-0 

231095 

ETHNICITY 

BY  GENDbR  (ASIAN  AMERICAN,  FEMALE) 

22 

E2  X  PE2 

0 

063586 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

23 

E2  X  PE3 

J 

375105 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  POST  HS) 

24 

E2  X  PE4 

0 

039552 

ETHi^ilCITY 

BY  PARENT'S  ED  (BLACK,  COLLFCR) 

25 

E2  X  PE 

0 

191412 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

26 

E3  X  PE2 

-0 

354255 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  HS  GR..D) 

27 

E3  X  PE3 

0 

237226 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC    POST  HS) 

28 

E3  X  PE4 

-0 

256883 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

29 

E3  X  PE 

-0 

246003 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

30 

E4  X  PE2 

-1 

034833 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AM,  HS  GRAD) 

31 

E4  X  PE3 

-0 

690193 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AM,  POST  HS) 

32 

E4  X  PE4 

-0 

786758 

ETHNICITY 

BY  PARENT'S  CD  (ASIAN  AM,  COLLEGE) 

33 

E4  X  PE_ 

-0 

518339 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AM,  UNKNOWN) 

34 

SCH  TYP2 

0 

158816 

SCHOOL  TYPE  (NOT  PUBLIC) 

35 

SCH  TYP 

SCHOOL  TYPE  (MISSING) 

36 

TVl 

0 

278883 

0-2  HOURS 

OF  TV  WATCHING 

37 

TV2 

0 

434684 

3-5  HOURS 

OF  TV  WATCHING 

38 

TV3 

0 

259356 

6+  HOURS 

OF  TV  WATCHING 

39 

LANGH0M3 

-0 

283533 

LANGUAGE  IN  HOME  OTHER  THAN  ENG  ^  (ALWAYS) 

40 

LANGH0M2 

0 

088718 

LANGUAGE  IN  HOME  OTHER  THAN  ENG.?  SOMETIMES 

41 

E2  X  lEl 

0 

143997 

ETHNICITY 

BY  LANGUAGE  IN  HOME  (BLACK,  OFTEN) 

(continued) 
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Table  C-38  (continued) 


Estimated  Effects  for  Mathematics  Trend  'Conditioning  Variables 
1988  Bridge  to  1986,  Age  9 

Estimated 

Variable            Effect  Description 

42  E2  X  TH2           0.080093  ETHNICITY  BY  LANG  IN  HOME  (BLACK,  SOMETIMES) 

43  E3  X  Lril           0.390581  ETHNICITY  BY  LANGUAGE  IN  HOME(HISPANIC, OFTEN) 

44  E3  X  LH2         -0.117348  ETHNICITY  BY  LANG  IN  HOME(HISPANIC , SOMETIMES) 

45  E4  X  LHl           0.411.867  ETHNICITY  BY  LANGUAGE  IN  HOME(ASIAN  AM, OFTEN) 

46  E4  X  LH2           0.238582  ETHNICITY  BY  LANG  IN  HOME(ASIAN  AM, SOMETIMES) 

47  TIME  ASS  TIME  OF  ASSM'T(APPLICABLE  FOR  Y17,  N/AY19) 

48  STUDYCMP         -0.057134  ARE  YOU  STUDYING  COMPUTERS?  B004501  (YES) 

49  DRACE2             -0.069875  DERIVED  RACE/ETHNICITY  (BLACK) 

50  DRACE3             -0.341651  DERIVED  RACF./ETHNICITY  (HISPANIC) 

51  DRACE4               0.185246  DERIVED  RACE/ETHNICITY  (ASIAN  AMERICAN) 


ERIC 
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Table  C-39 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 
1988  Bridge  to  1986,  Age  13 


Estimated 

Variable 

Effect 

1 

OVERALL 

-1.504811 

2 

GENDER2 

-0.228401 

3 

ETHNIC2 

-0.242682 

4 

ETHSIC3 

0.086195 

5 

ETHNIC4 

0.378006 

6 

ST0C3 

0.534516 

7 

STOCl 

0.298905 

8 

REGI0N2 

-0.121025 

9 

REGI0N3 

-0.063070 

10 

REGI0N4 

-0.107134 

11 

PARED2 

0.140058 

12 

PARED3 

0.197777 

13 

FARED4 

0.278975 

14 

PARED_ 

0.021061 

15 

<KODAL  GRADE 

-0.480949 

16 

>MODAL  GRADE 

0.541153 

17 

ITEHS2 

0.122176 

18 

ITEMS3 

0.177230 

19 

E2  X  SEX 

0.020985 

20 

E3  X  SEX 

0.099927 

21 

E4  X  SEX 

-0.0'  )259 

22 

E2  X  PE2 

-0  X81870 

23 

E2  X  PE3 

-0.179468 

24 

E2  X  PE4 

-0.397062 

25 

E2  X  PE_ 

0.090978 

26 

E3  X  PE2 

-0.033586 

27 

E3  X  PE3 

-0.035114 

28 

E3  X  PE4 

-0.359408 

29 

E3  X  PE 

-0.15CJ07 

30 

E4  X  PE2 

-0.412270 

31 

E4  X  PE3 

-1.023135 

32 

E4  X  PE4 

0.005724 

33 

E4  X  PE_ 

-0.148864 

34 

SCH  TYP2 

0.019369 

35 

SCH  TYP_ 

36 

TVl 

-0.192841 

37 

TV? 

-0.259867 

38 

TV3 

0.391540 

39 

HW-NO 

0.143508 

40 

HW-YES 

0.295564 

41 

HW-345 

-0.046762 

(continued) 

Description 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

GENDER  (FEMALE) 

OBSERVED  ETHNICITY  (BLACK) 

OBSEIUfiD  ETHNICITY  (HISPANIC) 

OBSERVED  ETHNICITY  (ASIAN) 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HI&NOT  IJO) 

REGION  (SOUTHEAST) 

REGION  (CENTRAL) 

REGION  (WEST) 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 
PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 
PARENTS  EDUCATION  (COLLEGE  GRAD) 
PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 
MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 
MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 
ITEMS  IN  THE  HOME  (YES  TO  3) 
ITEMS  IN  THE  HOME  ^YES  TO  ALL  4) 
ETHNICITY  BY  GENDER  (BLACK,  FEMALE) 
ETHNICir:  BY  GENDER  (HISPANIC,  FEMALE) 
ETHNICITY  BY  GENDER  (ASIAN  AMERICAN,  FEMALE) 
ETHNICITY  BY  PAREK.'S  ED  (BLACK,  HS  GRAD) 
ETHNICITY  BY  PARENT'S  ED  (BLACK,  POST  HS) 
ETHNICITY  BY  PARENT'S  ED  (BLACK,  COLLEGE) 
ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 
ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 
ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 
ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 
ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 
ETHNICITY  BY  PARENT'S  ED  (ASIAN  AM,  HS  GRAD) 
ETHNICITY  BY  PARENT'S  ED  (ASIAN  AM,  POST  HS) 


ETHNICITY  BY  PARENT'S  ED  (ASIAN  AM, 
ETHNICITY  BY  PARENT'S  ED  (ASIAN  AM, 
SCHOOL  TYPE  (NOT  PUBLIC) 
SCHOOL  TYPE  (MISSING) 
0-2  HOURS  OF  TV  WATCHING 
3-5  HOURS  OF  TV  WATCHING 
6+    HOURS  OF  TV  WATCHING 
HOMEWORK  (HONE  ASSIGi^ED) 
HOMEWORK  (YES  -  SOME  AMOUNT) 
HOMEWORK  (LINEAR  AMOUNT) 


COLLEGE) 
UNKNOWN) 


603 


6d2 


Table  C-39  (continued) 

Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 
1988  Bridge  to  1986,  Age  13 


Estimated 

Variable 

Effect 

42 

LANGK0M3 

-0 

.142210 

LANGUAl5E  IN  HOME  OTHFR  THAM  RNfiT  T<;H?  ^ATUAY<;\ 

43 

LANGH0M2 

0 

.050961 

44 

E2  X  LHl 

0 

.100579 

ETHNICITY  BY  TJVNfillAnF  IN  HOMF  CMACV  nTTTRK^^ 

45 

E2  X  LH2 

0 

.051984 

ETHNICITY  BY  LANG  IN  HOME  ^HI<;P  <;nMFTTMF<;^ 

ij  X 1111 X  v« X  X  X    u  X    Xjniiv?    All    iivyLiu    ^ ii X Ox  •  ,    OUnCi  X  ^nCiO  J 

46 

E3  X  LHl 

0 

.032823 

ETHNICITY  BY  LANGUAGE  IN  HOME  (HISP. ,  OFTEN) 

47 

E3  X  1H2 

-0 

.081489 

ETHNICITY  BY  LANG  IN  HOME  (HISP.,  SOMETIMES) 

48 

E4  X  LHl 

-0 

.295872 

ETHNICITY  BY  LANG  IN  HOME  (ASIAN  AM, OFTEN) 

49 

E4  X  LH2 

-0 

351225 

ETHN.  BY  LANG  IN  HOME  (ASIAN  AM, SOMETIMES) 

50 

GRADES 

0 

329379 

GRADES  IN  SCHOOL 

51 

TYPEMAT2 

0 

557133 

TYPE  OF  MATH  CLASS  (REGULAR  MATH) 

52 

TYPEMAT3 

0 

860079 

TYPE  OF  MATH  CLASS  (PRE -ALGEBRA) 

53 

TYPEMAT4 

1 

067878 

TYPE  OF  MATH  CLASS  (ALGEBRA,  OTHER) 

54 

TIME  ASS 

TIME  OF  ASSESSMENT  (APPLICABLE  Y17 ,  N/A  Y19) 

55 

STUDYCMP 

0 

000685 

ARE  YOU  STUDYING  COMPUTERS?  B004501  (YES) 

56 

DRACE2 

0 

021696 

DERIVED  RACE/ETHNICITY  (BLACK) 

57 

DRACE3 

-0 

262241 

DERIVED  RACE/ETHNICITY  (HISPANIC) 

58 

DRACE4 

0 

239560 

DERIVED  RACE/ETHNICITY  (ASIAN  AMERICAN) 
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Table  C-40 


Estii-dted  Effects  for  Mathematics  Trend  Conditioning  Variables 
1988  Bridge  to  1986,  Age  17 


Estimated 

V  CtX  XCiU 

Effect 

1 

X 

vIV  I!»tVALjlj 

0. 

466202 

0 

-0. 

;127644 

-0. 

326424 

A 
*+ 

•0. 

125207 

Lf  l£m  1U*4 

-0. 

542147 

C. 
D 

0. 

355679 

7 

O  1  uux 

0. 

268174 

Q 

o 

PR(^TnN9 

-0. 

035567 

0 

tvCV7  X  ULi  O 

0 

092946 

JLU 

0. 

041544 

PApRn9 

-0. 

009106 

0. 

276562 

PARRHZt 

0. 

215802 

PARRH 

0. 

039054 

X^ 

-0. 

,212266 

1  6 

,091063 

17 

ITEMS2 

0, 

.032057 

18 

ITEMS 3 

0, 

.089343 

19 

E2  X  SEX 

0, 

.130167 

20 

E3  X  SEX 

0, 

.294555 

21 

n4  X  SEX 

-0. 

.190247 

22 

E2  X  PS2 

-0. 

.014269 

^3 

E2  X  PE3 

-0. 

.186204 

F9  X  PE4 

-0, 

. 163440 

F9  X  PF 

-0, 

.256462 

F'^  X  PF9 

0, 

.037801 

97 

E3  X  PE3 

-G, 

.197622 

98 

F3  X  PE4 

-0 

. 148578 

99 

E3  X  PE 

0 

.076608 

30 

E4  X  PE2 

1 

.148569 

31 

E4  X  PE3 

0 

.548141 

32 

E4  X  PE4 

-0 

.003476 

33 

E4  X  PE_ 

0 

.555852 

34 

SCH  "TP2 

-0 

.130104 

35 

SCH  TYP 

36 

iVl 

-1 

.980878 

37 

TV2 

•1 

.992986 

38 

TV3 

-2 

.079726 

39 

HW-NO 

-0 

.243494 

40 

HW-YES 

0 

.104266 

41 

HW-345 

-0 

.024606 

(continued) 

Description 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

GENDER  (FEMALE) 

OBSERVED  ETHNICITY  (BLACK) 

OBSERVED  ETHNICITY  (HISPANIC) 

OBSERVED  ETHNICITY  (ASIAN) 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METR^ 

SIZE  AND  TYPE  OF  COMMONITY  (NOT  HI&NOT  LO) 

REGION  (SOUTHEAST) 

REGION  (CENTRAL) 

REGION  (WEST) 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 
PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 
PARENTS  EDUCATION  (COLLEGE  GRAD) 


parent:  education  (missing,  i 

MODAL  GRADE  (LESS  THAN  MODAL  ' 


DON'T  KNOW) 
3RADE) 

MODAL  GilADE  (GREATER  THAN  MODAL  GPJ^.DE) 
ITEMS  IN  THE  HOME  (YES  TO  3) 
ITEMS  IN  THE  HOME  (YES  TO  ALL  4) 
ETHNICITY  BY  GENDER  (BLACK,  FEMALE) 
ET.^IICITY  BY  GENDER  C'JSPANIC,  FEMALE) 
ETHNICITY  BY  GENDER  (ASIAN  AMERICAN,  FEMALE) 
ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 


ETHNICITY  BY  PARENT'S  ED 
ETHNICITY  BY  PARENT'S  ED 
ETHNICITY  BY  PARENT'S  ED 
ETHNICITY  BY  PARENT'S  ED 
ETHNICITY  BY  PARENT'S  ED 
ETHNICITY  3Y  PARENT'S  ED 
ETHNICITY  BY  PARENT'S  ED 
ETHNICITY  BY  PARENT'S  ED 
ETHNICITY  BY  PARENT'S  ED 
ETHNICITY  BY  PARENT'S  ED 
ETHNICITY  BY  PARENT'S  ED 
SCHOOL  TYPE  (NOT  PUBLIC) 
SCHOOL  TYPE  (MISSING) 
0-2  HOURS  OF  TV  WATCHING 
3-5  HOURS  OF  TV  WATCHING 
6+  HOURS  OF  TV  WATCHING 
HOMEWORK  (NONE  ASSIGNED) 
HOMEWORK  (YES  -  SOME  AMOUNT) 
HOMEWORK  (LINEAR  AMOUNT) 


(BLACK,  POST  HS) 
(BLACK,  COLLEGE) 
(BLACK,  UNKNOWN) 
(HISPANIC,  HS  (;RAD) 
(HISPANIC,  POST  HS) 
(HISPANIC,  COLLEGE) 
(HISPANIC,  UNKNOWN) 
(ASIAN  AM,  HS  GRAD) 
(ASIAN  .\M,  POST  HS) 
(ASIAN  AM,  COLLEGE) 
(ASIAN  AM,  UNKNOWN) 


ERLC 
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Table  C-40  (continued) 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 
1988  Bridge  to  1986,  Age  17 


Estimated 

Variable 

Effect 

Description 

42 

LANGH0H3 

-0.306630 

LANGUAGE  IN  HOKE  OTHER  THAN  ENGLISH?  (ALWAYS) 

43 

LANGH0H2 

-0.027324 

LANGUAGE  IN  HOME  OTHER  THAN  ENG.?  (SOMETIMES) 

44 

E2  X  LHl 

0.234334 

ETHNICITY  BY  LANGUAGE  IN  HOME  (BLACK,  OFTEN) 

45 

E2  X  LH2 

-0.085786 

ETHNICITY  BY  LANG  IN  HOME  (HISP,  SOMETIMES) 

46 

E3  X  LHl 

0.372056 

ETHNICITY  BY  LANGUAGE  IN  HOME  (HISP,  OFTEN) 

47 

E3  X  LH2 

0.068137 

ETHNICITY  BY  lANG  IN  HOME  (HISP,  SOMETIMES) 

48 

E4  X  LHl 

0.542742 

ETHNICITY  BY  LANG  IN  HOME  (ASIAN  AM,  OFTEN) 

49 

E4  X  LH2 

0.390736 

ETHN.  BY  LANG  IN  HOME  (ASIAN  AM, SOMETIMES) 

50 

NMATHl 

-0.221100 

HIGHEST  LEVEL  MATH  TAKEN  (PRE- ALGEBRA) 

51 

NMATH2 

0.252774 

HIGHEST  LEVEL  MATH  TAKEN  (ALCrBRA) 

52 

NMATH3 

0.354687 

HIGHEST  LEVEL  MATH  TAKEN  (GEOMETRY) 

53 

NMATH4 

0.700470 

HIGHEST  LEVEL  MATH  TAKEN  (ALGEBRA- 2) 

54 

NhATH5 

1.2C8891 

HIGHEST  LEVEL  MATH  TAKEN  (CALCULUS) 

55 

COMPUTER 

-0.009892 

COMPUTER  CLASS  TAKEN  ?  (YES) 

56 

GRADES 

0.293596 

GRADES  IN  SCHOOL 

57 

HSPR0G2 

0.196396 

HIGH  SCHOOL  PROGRAM  (COLLEGE  PREP) 

58 

HSPR0G3 

-0.09002? 

HIGH  SCHOOL  PROGRAM  (VOC/TECH) 

59 

DRACE2 

0.119675 

DERIVED  RACE/ETHNICITY  (BLACK) 

60 

DRACE3 

-0.202548 

DERIVED  RACE/ETHNICITY  (HISPANIC) 

61 

DRACE4 

-0.056777 

DERIVED  RACE/ETHNICITY  (ASIAN  AMERICAN) 
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Table  C-41 


Estimated  Ef^3cts  for  Science  Trend  Conditioning  Variables 
1988  Bridge  to  1906,  Age  9 


Estimated 

Variable 

Effect 

Description 

L 

OVERALL 

-0. 

167629 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

-0. 

160032 

GENDER  (FEMALE) 

3 

ETHNIC2 

-0. 

716027 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC3 

-0. 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

-0. 

143962 

OBSERVED  ETHNICITY  (ASIAN) 

6 

ST0C2 

-0. 

400385 

SIZE  AND  TYPE  OF  COMMUNITY  (LOW  METRO) 

7 

ST0C3 

0. 

114765 

SIZE  AND  TYPi:.  OF  COMMUNITY  (HIGH  METRO/ 

8 

REGI0N2 

0. 

105314 

REGION  (SOUTHEAST) 

9 

REGI0N3 

0 

202669 

REGION  (CENTRAL) 

10 

REGI0N4 

0 

081810 

REGION  (WEST) 

11 

PARED2 

J 

200699 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0 

279235 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

435635 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

0 

null 

PARENTS  EDUCATION  (MISSING,  DON'T  ^NOW) 

15 

<MODAL  GRADE 

-0 

.498134 

MODAL  GRADE  (LESS  THAN  MODAL  GRADC) 

16 

>MODAL  GRADE 

1 

.050936 

MODAL  GRADE  (GREATER  THAN      *JAL  GRADE) 

17 

ITEMS2 

0 

.289243 

ITEMS  IN  THE  HOME  (YES  TC   •  • 

18 

ITEMS 3 

0 

ITEMS  IN  THE  HOME  (YES  TO  ALL  4) 

19 

SCH  TYP2 

0 

.076284 

SCHOOL  T^PE  (NOT  PUBLIC) 
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Table  0-42 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 
1988  Bridge  to  1986,  Age  13 


Estimated 

Variable 

Effect 

Description 

1 

OVERALL 

-0.048884 

OVERALL  CONSTANT  ' 1 '  FOR  EVERYONE 

2 

GENDER2 

-0.267412 

GENDER  (FEMALE) 

3 

ETHNIC2 

-0.719052 

OBSFRVED  ETHNICITY  (BLACK) 

4 

ETHNIC3 

-0.524609 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

0.161636 

OBSERVED  ETHNICITY  (ASIAN) 

6 

ST0C2 

-0.395130 

SIZE  IBD  TYPE  OF  COMMUNITY  (LOW  METRO) 

7 

ST0C3 

-0.007911 

SIZE  /ND  TYPE  OF  COMMUNITY  (HIGH  MbiRO; 

8 

REGI0N2 

-0.077003 

REGION  (SOUTHEAST) 

9 

REGI0N3 

0.046762 

REGION  (CENTRAL) 

10 

REGI0N4 

-0.102571 

REGION  (WEST) 

11 

PARED2 

0.1 J7733 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED 3 

0. 3^7308 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0.412279 

PAPvENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

-0.047971 

PARENTS  EDUCATION  (MISSING,  DON'T  KNOW) 

15 

<MODAL  GRADE 

-0.530171 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>MODAL  GRADE 

0.969538 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

ITEMS2 

0.222418 

ITEMS  IN  THE  HOME  (YES  TO  3) 

18 

ITEMS 3 

0.404732 

ITEMS  IN  THE  HOME  (YES  TO  ALL  4) 

19 

SCH  TYP2 

0.128735 

SCHOOL  TYPE  (NOT  PUBLIC) 
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Table  C-43 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 
1988  Bridge  to  1986.  Age  17 


Estimated 
Variable  Effect 


1 

OVERALL 

-0  018353 

2 

GENDER2 

-0.422265 

3 

FTH^'IC2 

-0.675393 

4 

ETHNIC 3 

-0.028940 

5 

FTHNT  CM 

0. 105174 

6 

ST0C2 

-0.215624 

7 

ST0C3 

0.200910 

8 

REGI0N2 

-0.078230 

9 

REGI0N3 

-0.145136 

10 

REGI0N4 

-0.156447 

11 

PARED2 

0.277744 

12 

PARED3 

0.506933 

13 

PARED4 

0.724225 

14 

PARED_ 

-0.353136 

15 

<MODAL  GRADE 

-0.540566 

16 

>MODAL  GRADE 

0.345666 

17 

ITEMS2 

0.091730 

18 

ITEMS3 

0.208488 

19 

SCH  TYP2 

-0.094395 

Description 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

GENDER  (FEMALE) 

OBSERVED  ETHNICITY  (BLACK) 

OBSERVED  ETHNICITY  (HISPANIC) 

OBSERVED  ETHNICITY  (ASIAN) 

SIZE  AND  TYPE  OF  COMMUNITY  (LOW  METRO) 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

REGION  (SOUTHEAST) 

REGION  (CENTRAL) 

REGION  (WEST) 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 
PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 
PARENTS  EDUCATION  (COLLEGE  GRAD) 
PARENTS  EDUCATION  (MISSING,  DON'T  KNOW) 
MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 
MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 
ITEMS  IN  THE  HOME  (YES  TO  3) 
ITEMS  IN  THE  HOME  (YES  TO  ALL  4) 
SCHOOL  TYPE  (NOT  PUBLIC) 
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Appendix  D 

1988  NAEP  COMPOSITE  AND  DERIVED  CONDITIONING  VARIABLES 


Some  conditioning  variables  used  in  the  creation  of  plausible  values 
were  constmicted  by  recoding  the  values  of  a  data  variable  or  by  combining  and 
recoding  data  from  two  or  more  variables.    This  appendix  describes  how  these 
conditioning  variables  were  derived  for  the  1988  assessments  of  reading 
writing,  civics,  U.S.  history,  geography,  mathematics  and  science. 


Reading  Cross- sectional  Variables 

Seven  composites  of  background,  attitude,  and  instructional  variables 
were  created  for  use  as  conditioning  variables  for  cross- sectional  studies  in 
reading.    The  following  list  shows  the  items  included  in  the  composites.  In 
some  cases,  the  constituent  items  vary  across  age  cohorts,  as  indicated.  In 
each  case,  items  within  a  composite  were  reflected  so  that  they  were  coded  "in 
the  same  direction"  (e.g.,  more  frequent  use  of  a  teaching  technique  received 
a  higher  score  than  less  frequent  use,  etc.).    The  composite  score  for  each 
student  is  a  weighted  sum  of  the  item  scores.     (Weights  were  based  on 
principal  component  analyses  of  each  set  of  items;  the  weights  were  those 
associated  with  the  first  principal  component.)    Students  with  scores  more 
than  half  a  standard  deviation  above  the  mean  on  the  composite  v/ere  considered 
"high,"  those  with  scores  more  than  half  a  standard  devia'  Ion  below  the  mean 
were  considered  low,  and  the  remaining  students  were  considered  "medium." 
Composite  scores  were  not  calculated  for  students  who  responded  to  fewer  than 
two  of  the  constituent  items.    If  the  student  responded  to  at  least  two  items 
but  did  not  respond  to  all  items  in  the  composite,     the  composite  score  was 
taken  to  be  the  mean  on  the  completed  items.    See  Table  C-2  ("Contrast  Codings 
for  1988  Reading  Cross-sectional  Conditioning  Variables")  for  the  coding  of 
the  constituent  items.  (Missing  responses  were  treated  as  stated  above  and  not 
recoded  as  indicated  in  Table  C-2.) 


RHOME  (Home  environment  and  support) 


R80J101 
R800201 
R800301 
R800401 


Magazines  at  home 
Magazines  at  home 
Books  at  home 
Read  stories 


Ages 

9 

13  17 


9  13  17 
9    13  17 
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RINDRDG  (Students'  independent  reading) 


Ages 


R800601 

Talk  about  reading 

1  Q 

1  "7 

1/ 

S003501 

Read  for  fun 

9 

13 

17 

Su^350Z 

Talk  about  book 

0 

1/ 

S003503 

Books  from  library 

9 

13 

17 

S003504 

Buy  own  books 

9 

13 

17 

S003505 

Read  after  TV  show 

13 

17 

S003506 

Author  you  like 

13 

17 

LSOllOl 

Read  in  spare  time 

13 

17 

R800701 

Read  stories 

9 

S004301 

Read  story, novel 

17 

RUSEMAT  (Students'  use  of  materials) 

R800801  Use  dictionary 

S007309  Use  encyclopedia 

5004304  Read  newspaper 

5004305  Read  magazine 


Ages 
13  17 
13  17 
17 
17 


RTEACH  (Teacher  instructional  behaviors) 

S004602  Teacher-main  idea 

S004601  Teacher-vocabulary 

S004701  Teacher-questions 

R801201  Analyze  reading 

S008501  Write  on  reading 


Ages 
13  17 
13  17 
13  17 
13  17 
13  17 


RBEHAV  (Students'  behaviors) 

Ages 

R800901  Time  on  literature  17 
R801001         Time  on  homework  17 


RSTRATG  (Reading  strategies) 

5005101  Read  material  over 

5005102  Take  notes 

5005103  Make  outlines 

5005104  Answer  questions 


Ages 
17 
17 
17 
17 


RSCHWRK  (Students'  school/coursework) 

Ages 

R801101  English  grades  17 
S006403  Advanced  placement  English  17 
S006401         Remedial  English  17 
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Writing  Cross -sectional  Variables 


WENJOY    (Enjoy  Writing) 

For  grades  4,  8,  and  12,  responses  to  item  S001201  were  recoded  as 
follows : 

Always       -  5 

Half  +       -  4 

Half          -  3 

Half  -       -  2 

None          -  1 

Missing     -  3 


WPOSFB  (Positive  Teacher  Feedback) 

For  grades  4,  8,  and  12,  responses  to  items  S001703,  S001706,  and 
S002503  were  recoded  as  indicated  below  and  then  summed.    Values  ranged  from  0 
to  3, 

Mostly  all    -  1 


Half  +  -  1 

Half  -  0 

Half  -  -  0 

Never  -  0 

Missing  -  0 


WNEGFB  (Negative  Teacher  Feedback) 

For  grades  4,  8,  and  12,  responses  to  items  S001708  and  S002505  were 
recoded  as  indicated  below  and  then  summed.    Values  ranged  from  0  to  2. 

Mostly  all    -  0 


Half  +  -  0 

Half  -  0 

Half  -  -  1 

Never  -  1 

Missing  -  0 


615 


WAMTWE  (Amount  Written) 


For  grades  8  and  12,  responses  to  items  W800302  (1  or  2  pages)  and 
W800303  (3  pages)  were  recoded  as  follows: 

Daily  -  1 

Weekly  -  2 

Monthly  -  3 

Yearly  -  4 

Never  -  5 

Missing  -  Missing 

The  variables  were  then  coded  as  six  dummy  variables  as  follows: 

V800302  W800303  Dummy  Variables 

1,2  5  100000 

1,2  1,?  010000 

1,2  3,4  001000 

3, 4, missing  3, 4, missing  000100 

3,4  5  000010 

5  5  000000 

5  1,2,3,4  000001 

3,4  1,2  000001 


WAMTWH  (Writing  in  History  Class) 

For  grades  8  and  12,  responses  to  items  W800802  (1  or  2  pages)  and 
W800803  (3  pages)  were  recoded  as  follows: 

Daily  -  1 

Weekly  -  2 

Monthly  -  3 

Yearly  -  4 

Never  -  5 

Missing  -  Missing 

The  variables  were  then  coded  as  six  dummy  variables  as  follows: 

W800802  W800803  Dummy  Variables 

1,2  5  100000 

1,2  1,2  010000 

1,2  3,4  001000 

3,4, missing  3 , 4 , missing  000100 

3,4  5  000010 

5  5  000000 

5  1,2,3,4  000001 

3,4  1,2  000001 
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WINST2  (Instruction  on  Writing  Process  II) 


For  grades  8  and  12,  responses  to  items  S000606,  S000608,  S000609,  and 
S000610  were  receded  as  indicated  below;  the  receded  response?  for  the  four 
iteros  were  then  averaged. 

Always  -  5 

Half  +  -  4 

Half  -  3 

Half  -  -  2 

Never  -  1 


WSUC2  (Success  in  English  Class  II) 

For  grades  8  and  12,  responses  to  item  S008601  were  coded  as  follows: 

A  lot  -  1 

Some  -  2 

None  -  3 

Missing  -  2 


WSUC3  (Success  in  English  Class  III) 

For  grade  1*^    responses  to  item  W800701  were  recoaed  as  follows: 


Most  A 

8 

Half  A-B 

mm 

7 

Most  B 

6 

Half  B-C 

mm 

5 

Most  C 

mm 

4 

Half  C-D 

mm 

3 

Most  D 

2 

Most  <  D 

1 

Missing 

5 

WTIMSP3  (Time  Spent  III) 

For  grade  12,  responses  to  four  items  were  recoded  as  indicated  below. 
The  recoded  responses  to  all  four  items  were  then  av^eraged.  If  responses  to 
all  four  items  wtre  missing,  WTIMSP3  was  given  a  value  of  2. 

Codings 

Weekly  4 

Monthly  -  3 

Yearly  -  2 

Never  -  1 
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Items  Used 

W800501 

W800502 
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W800601  None  -  2 

Don't  do  -  1 

<  1  hour  «  2 

1  hour  -  3 

2  hour  -  4 
3+  hours  -  4 

S000201  None  -  1 

1  to  2  -  2 

3  to  4  -  3 
5  to  10  -  4 
11+  -  4 


WREVISE  (Revision) 

For  grade  12,  responses  to  items  S001302,  S001303,  S001304,  S001308,  and 
S001301  were  recoded  as  indicated  below.    The  receded  responses  to  all  five 
items  were  then  averaged.    If  responses  to  all  five  items  were  missing, 
WREVISE  was  given  a  value  of  3. 


Always  -  5 

Half  +  -  4 

Half  -  3 

Kalf  -  -  2 

Kever  -  1 


WPLANITG  (Planning) 

For  grade  12,  responses  to  items  S000901  and  S000902  were  recoded  as 
indicated  below.    The  recoded  responses  to  both  items  were  then  averaged.  If 
responses  to  both  itams  were  missing,  WPLANNG  was  given  a  value  of  3. 


Host,  all 

5 

Half  + 

4 

Half 

3 

Half  - 

2 

Never 

1 

WTIHSP2  (Time  Spent  II) 

For  grade  8,  responses  to  item  W800502  were  recoded  as  follows: 

Weekly  -  4 

Monthly  -  3 

Yearly  -  2 

ilftver  -  1 

Missing  -  3 
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WriMSPl  (Time  Spent  I) 


For  grade  4,  three  dummy  variables  were  created  from  the  responses  to 
Item  W8002ul  as  follows: 


1 

Weekly 

100 

2 

Monthly 

010 

3 

Yearly 

001 

4 

Never 

000 

Hissing 

010 

WSUCl  (Success  in  English  Class  I) 

For  grade  4,  four  dummy  variables  were  created  from  the  responses  to 
item  W800401  as  follows: 


1 

Dally 

1000 

2 

Weekly 

0100 

3 

Monthly 

0010 

4 

Yearly 

0001 

5 

Never 

OOOC 

Missing 

0010 

WINSTl  (Instruction  on  Writing  Process  1) 

For  grade  4,  responses  to  items  5000604  and  S000605  were  recoded  as 
indicated  below.    The  recoded  responses  to  both  items  were  then  averaged.  If 
responses  to  both  items  were  missing,  WINSTl  was  given  a  value  of  3. 

Always  -  5 

Half  +  -  4 

Half  -  3 

Half  -  -  2 

Nev^r  -  1 


Writing  Trend  Variables 

NUMREP  (Number  of  Reports  Written  for  English  Class) 

For  grades  8  and  12,  if  the  student  had  an  English  class  (i.e.,  response 
to  items  B001208  is  not  1),  responses  to  items  B001201 -5001207  were  recoded  as 
follows: 

None  -  0 

1  or  2  written  -  1 

3  or  more  written  -  1 
BLK  missing  -  0 
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The  sua  of  the  seven  responses  was  then  computed  and  the  number  of 
reports  written  for  English  class  became  0  (for  0  reports  or  missing)  or  1  to 
7  (for  1  to  7  reports). 


Civics  Cross- sec tictial  Variables 

ST-ALOT  (Number  of  Subjects  F  :udied  a  Lot) 

For  grade  4,  items  P800601,  P800701,  P800801,  P800901,  P801001,  and 
P801101  asked  students  to  indicate  whether  they  had  studied  particular  civics 
relatvjd  topics  a  lot,  some,  or  not  at  all.     Each  of  the  six  items  was  coded  as 
follows : 


A  lot  -  1 

Some  -  0 

Not  at  all  -  0 
Hissing       -  0 


The  sum  of  the  six  recoded  variables  was  defined  as  ST-ALOT, 


STCV-INC  (Studied  Civics — Inccmplete  Set  of  Answers) 

For  grade  8,  items  P800101  to  P800104  asked  students  whether  they  had 
studied  American  government  or  jivics  in  grades  5  to  8,  respectively.    Each  of 
the  four  items  was  coded  as: 


Yes  -  0 

No  -  0 

I  don' t  know  -  1 
Missing  -  1 


The  sum  of  the  four  variables  was  then  assigned  the  codes: 

0  -  0 
1«4  -  1 


ST-ALOT  (Number  of  Subjects  Studied  a  :^t> 

For  grade  8,  items  P801201  to  P801210  asked  students  to  in'iicate  whether 
they  had  studied  particul£.^  civics- related  topics  a  lot,  some  or  not  at  all. 
Each  of  the  ten  items  was  cod^d  as  follows: 


A  lot  1 

Some  ^  0 

Not  at  all  «  0 

Missing  -  0 


The  sum  of  the  ten  recoded  variables  was  defined  as  ST-ALOT. 
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CVA-ACT  (Active  in  4  Areas  of  Civics,  1,  2,  5,  and  8) 


For  grade  8,  items  P802201,  P802202,  P802205,  and  .P802208  asked  students 
to  indicate  how  often  they  participated  in  particular  educational  activJties 
for  social  studies  class.    These  items  were  grouped  together  because  of  the 
similarity  of  responses  to  them.    Each  of  the  four  items  was  reccdod  as: 

Almost  every  day  -  1 

Otice  or  twice  a  week  -  1 
Once  or  twice  a  month  -  0 
A  few  times  a  year  -  0 
Never  -  0 

Missing  «  0 

The  sum  of  the  four  variables  was  *  ^n  assigned  the  codes: 

4  -  1 
0-3  -  0 


CVli-ACT  Cft'^tive  in  2  Areas  of  Civics,  3  and  10) 

For  grade  8,  item?  P802203  and  PSC2210  asked  students  to  indicate  how 
often  they  particip^*  ed  in  particular  educational  activities  for  social 
studies  class.    These  items  were  grouped  together  because  of  the  similarity  of 
responses  to  them.    Each  of  the  two  items  was  recoded  as: 

Almost  every  day  -  1 

Once  or  twice  a  week  -  1 
Once  or  twice  a  month  -  0 
Never  -  0 

Missing  -  0 

The  sum  of  the  two  '^'ariables  was  then  assigned  the  codes: 

2  -  1 
0-1  -  0 
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CVC-ACT  (Active  in  2  Areas  of  Civics,  4  and  9) 


For  grade  8,  items  P802204  and  P802209  asked  students  to  indicate  how 
often  they  participated  in  particular  educational  activities  for  social 
studies  class.    These  items  were  grouped  together  because  of  the  similarity  of 
responses  to  them.    Each  of  the  two  items  was  recoded  as: 

Almost  every  day  -  1 

Once  or  twice  a  week  -  1 
Once  or  twice  a  month  -  0 
A  few  times  a  year  -  0 
Never  -  0 

Missing  -  0 

The  sum  of  the  two  variables  was  then  assigned  the  codes: 

2  -  1 
0-1  -  0 


CVD-ACT  (Active  in  2  Areas  of  Civics,  6  and  7) 

For  grade  8,  items  P802206  and  P802207  asked  students  to  indicate  how 
often  they  participated  in  particular  educational  activities  for  social 
studies  class.    These  items  were  grouped  together  because  of  the  similarity  of 
responses  to  them.     Each  of  the  two  items  was  recoded  as: 

Almost  every  day  •»  1 

Once  or  twice  a  week  -  1 
Once  or  twice  a  month  -  0 

A  few  times  a  year  -  C 

Never  il 
Missing 

The  sum  of  the  two  variables  was  then  assigned  the  codes: 

2  -  1 
0-1  -  C 

CVD-NOT  (Not  very  Active  in  these  2  Areas  of  Civics) 

For  grade  8,  items  P802206  and  P802207  asked  students  to  indicate  how 
often  chey  participated  in  particular  educational  activities  for  social 
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studies  class.  These  items  were  grcuped  together  because  of  the  similarity  of 
responses  to  them.    Each  of  the  two  items  was  recoded  as: 


Almost  every  day  -  0 

Once  or  twice  a  week  -  0 
Once  or  twice  a  month  -  0 

A  few  times  a  year  -  1 

Never  -  1 

Missing  -  1 

The  sum  of  the  two  variables  was  then  assigned  the  codes: 

2  -  1 
0-1  -  0 


STCV-INC  (Studied  Civics -Incomplete  set  of  Answers) 

For  grade  12,  items  P800201  to  P800204  asked  students  whether  they  had 
studied  American  government  or  civics  ii:  gr^.des  9  to  12,  respectively.  Each 
of  the  four  items  was  coded  as: 

Yes  -  0 

No  -  0 

I  don't  know  -  1 

Missing  -  1 

The  sum  of  the  four  variables  was  then  assigned  the  codes: 

0  -  0 
1-4    -  1 


GOVT  (How  Many  Years  of  Government  Study  have  You  Had) 

For  grade  12,  item  P800301  asked  for  the  number  of  years  the  student  had 
studied  American  government  or  civics  since  the  beginning  of  ninth  grade.  The 
responses  were  recoded  as: 


None  -  0 

Less  than  1/2  year  -  .25 

1/2  year  -  .5 
Between  1/2  year  and  1  year    -  .75 

1  year  -  1.0 

More  than  1  year  -  2.0 

Missing  -  0 


ERJC 


623 


660 


ST-ALOT  (Number  of  Subjects  Studied  a  Lot) 


For  grade  12,  items  P801301  to  P801310  asked  students  to  indicate 
whether  the>  had  studied  particular  civics -related  topics  a  lot,  some  or  not 
at  all.     Each  of  the  ten  items  was  coded  as  follows: 

A  lot  -  1 

Some  -  0 

Not  at  all  -  0 
Missing       -  0 

The  sum  of  the  ten  recoded  variables  was  defined  as  ST-ALOT. 


ST-AVE  (Average  Leval  of  Study  for  These  Ten) 

For  grade  12,  responses  to  items  P8O1301  to  P801310  were  recoded  as: 

A  lot  -  2 

Some  -  2 

Not  at  all  -  1 
Missing        -  0 

The  average  of  the  ten  recoded  variables  was  defined  as  ST-ALOT. 


CIV- INT  (Level  of  Interest  in  Civics) 

For  grade  12,  item  P801401  asked  students  how  much  they  like  studying 
American  government  or  civics.    The  responses  were  recoded  as: 

One  of  my  favorite  subjects  -  3 

In  teres  t5.ng  -  2 

Like  several  other  subjects  better  -  1 

Never  studied  -  0 


GOVATTN  (Degree  of  Attention  Government  Pays) 

For  gradr  12,  items  P801601  and  P801701  asked  students  to  give  their 
opinions  about  the  responsiveness  of  government  to  the  people.    Each  of  the 
two  items  was  coded  as: 

A  good  deal  -  2 
Some  -  1 

Not  much         -  0 

The  average  of  the  two  variables  was  defined  as  GOVATTN. 
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The  responses  to  HADCIV  (yes,  have  had  a  civics  class)  influenced 
the  coding  of  the  following  variables.    This  item  (P801801)  asked 
students  if  they  had  ever  had  an  American  government  or  civics  class. 
If  they  had,  they  were  to  continue  answering  questions  in  the  civics 
background  variable  block.    If  not,  they  :^ere  asked  to  stop  If 
students  continued  to  answer  questions  although  they  had  not  said  that 
they  had  taken  a  civics  couise,  ^he  rest  of  the  responses  were  coded  as 
missing. 


CVHWORK  (Amount  of  Civics  Homework) 

For  grade  12,  responses  to  i":em  P801901  were  receded  as: 


No  assignments  -  0 

Don't  do  assignment  -  0 

Ijsss  than  1  hour  -.5 

1  hour  -  1 

2  hours  -  2 

3  hours  3 

4  hours  -  4 

5  hours  or  more  -  5 
Did  not  say  that  Civics  was  taken  -  0 
Missing  -  0 


CVGRADES  (Civics  Grades  in  School) 

For  grade  12,  responses  to  item  P802001  were  receded  as: 


Mostly  A  -  4.0 

Half  A  and  half  B  -  3.5 

Mostly  B  -  3.0 

Half  B  and  half  C  -  2.5 

Mostly  C  «  2.0 

Half  C  and  hdlf  D  -  1.5 

Mostly  D  -  1.0 

Mostly  below  D  -  .5 

Did  not  say  that  civics  had  been  taken    -  2.0 

Missing  -  2.0 


CVA-ACT  (Active  in  4  Areas  of  Civics,  1,        5,  and  8) 


For  grade  12,  items  P802401,  P802402,  P802405,  and  P802408  asked 
students  to  indicate  how  often  they  participated  in  par. icular  educational 
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acclvities  for  civics  class.  These  items  were  grouped  together  because  of  the 
similarity  of  responses  to  them.     Each  of  the  four  items  was  recoded  as: 


Almost  every  day  ~  1 

Once  or  twice  a  week  -  1 

Once  or  t>7ice  a  month  -  0 

A  few  times      year  -  0 

Never  -  0 
Did  not  say  that  civl-^s  was  taken  -  0 

Missing  «  0 

The  sum  of  the  four  variables  was  then  assigned  the  codes: 


4     -  ^ 

0-3  -  J 


CVB-ACT  (Active  in  2  Areas  of  Civics,  3  and  10) 

For  grade  12,  items  P802403  and  P802410  asked  students  to  indicate  how 
often  they  participated  in  particular  educational  activities  for  civics  class. 
These  items  were  grouped  together  because  of  the  similarity  of  responses  to 
them.     Each  of  the  two  items  was  recoded  as: 


Almost  every  day  -  1 

Once  or  twice  a  week  -  1 

Once  or  twice  a  montb  -  0 

A  few  times  a  year  -  C 

Never  -  0 
Did  not  say  that  civics  was  taken  -  0 

Missing  -  0 


The  sum  of  the  two  variables  was  then  assigned  the  codes: 

2  -  1 
0-1  -  0 


CVC-ACT  (Active  in  2  Areas  of  Civics,  4  and  9) 

For  grade  12,  items  P802^04  and  P802409  asked  students  to  indicate  how 
often  they  participated  in  particular  educational  activities  for  civics  class. 
These  items  were  grouped  together  because  of  the  similarity  of  responses  :.hem. 
Each  of  the  two  items  was  recoded  as: 


Almost  evvsry  day  -  1 

Once  or  twice  a  week  -  1 

Once  or  twice  a  month  -  0 

A  few  times  a  year  -  0 

Never  -  0 

Did  not  say  that  civics  was  taken  -  0 

Missing  -  0 
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The  stun  of  the  two  variables  was  then  assigned  the  codes: 


2 

0-1 


-  1 

-  0 


CVD-ACT  (Active  in  2  Areas  of  Civics,  6  and  7) 

For  grade  12,  items  P802406  and  P802407  asked  students  to  indicate  how 
often  they  participated  in  particular  educational  activities  for  civics  class 
These  items  were  grouped  together  because  of  the  similarity  of  responses  to 
them.    Each  of  the  two  items  was  recoded  as: 


CVD-NOT  (Not  Very  Active  in  These  2  Areas  of  Civics) 

For  grade  12,  items  P802406  and  P802407  asked  students  to  indicate  how 
often  they  participated  in  particular  educational  activities  for  civics  class 
These  items  were  grouped  together  because  of  the  similarity  of  responses  to 
them.    Each  of  the  two  items  was  recoded  as: 


Almost  every  day 
Once  or  twice  a  week 
Once  or  twice  a  month 
A  few  times  a  year 
Never 

Did  not  say  that  cJ.vics  was  taken 
Missing 


-  1 

-  1 

-  0 

-  0 

-  0 

-  0 

-  0 


The  stun 


of  the  two  variables  was  then  assigned  the  codes: 


2  -  1 
0-1  -  0 


Almost  every  day  -  0 

Once  or  twice  a  week  -  0 

Once  or  twice  a  month  -  0 

A  few  times  a  year  -  1 

Never  -  1 
Did  not  say  that  civics  was  taken  -  1 

Missing  -  1 


The  stun  of  the  two  variables  was  then  assigned  the  codes: 


2  -  1 
0-1  -  0 
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RDDIF  (At  Least  Some  Difficulty  in  Reading  or  Missing) 


For  grade  12,  responses  to  item  P802301  were  recoded  as: 

A  lot  -  1 

Some  -  1 

None  -  0 

Did  not  say  that  civics  was  taken    -  1 

Missing  -  1 


Civics  Trend  Variables 

PPARED  (Parent's  Education  -  2nd  Set  of  Categories) 

For  ages  13  and  17,  a  parents'  education  variable  was  derived  for  1982 
data,  so  that  comparisons  with  1988  data  be  made  for  students  having  a  parent 
who  graduated  from  college.    This  variable  is  based  on  the  1982  variables 
PARED,  FCLGRAD,  and  MCLGRAD  (whether  the  father  or  mother  graduated  from 
college).     The  categories  for  PPARED  are  coded  as: 

1  <  High  school  if  PARED  -  1  (<HS) 

2  High  school  graduate  if  PARED  -  2  (HS  Grad) 

3  Post  high  school         if  PARED  -  3  (post-HS),  FCLGRAD  -  2(No), 

and  MCLGRAD  -  2  (No) 

4  College  graduate         if  PARED  -  3  and  either  FCLGRAD  -  1  (Yes) 

or  MCLGRAD  -  1  (Yes) 
Missing  if  otherwise 


U.S«  History  Cross-sectional  Variables 

HIS  TAKE  (How  often  do  you  have  a  social  studies  class?) 

For  grade  4,  responses  to  item  S008701  were  recoded  as  follows: 

Neve*:,  hardly  ever  or  missing  -  0 

Less  than  once  a  week  -  1 

One  or  two  times  a  week  -  2 

Three  or  four  times  a  week  -  3 

Every  day  -  4 
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STUD. HIS  (Have  you  ever  studied  the  history  of  our  country?) 

For  grade  4,  responses  to  item  H800601  wf^re  recoded  as  follows: 

No  or  missing  -  0 
Yes  -  1 

STUD. EXP    (Have  you  studied  the  early  explorers  of  the  new  world?) 

For  grade  4,  responses  to  item  H801101  were  recoded  as  follows: 

Not  at  all  or  missing  -  0 
Some  "  1 

A  lot  -  2 

STUD.IND       (Have  you  studied  the  Indians  who  first  lived  in  our  country?) 

For  grade  4,  responses  to  item  H801201  were  recoded  as  follows: 

Not  at  all  or  missing  «  0 
Some  "  1 

A  lot  -  2 

STUD.PIL       (How  much  have  you  studied  about  the  Pilgrims  and  the  first 
colonies  in  America?) 

For  grade  4,  responses  to  item  H801301  were  recoded  as  follows: 

Not  at  all  or  missing  -  0 
Some  -  1 

A  lot  -  2 

STUD.OvJ         ''How  much  have  you  studied  George  Washington  and  the  war  to  gain 
independence  irrom  England?) 

For  grade  4,  respovises  to  item  H801401  were  recoded  as  follows: 

Not  at  all  or  missing  -  0 
Some  -  1 

A  lot  -  2 
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STUD.PIO        (How  much  have  you  studied  the  pioneers  who  settled  the  western 
part  of  the  country?) 

For  grade  4,  responses  to  item  H801501  were  recoded  as  follows: 

Not  at  all  or  missing  -  0 
Some  -  1 

A  lot  "  2 


STUD.SIA        (How  much  have  you  studied  slavery  and  the  wa-  between  the 
Northern  and  Southern  states?) 

For  grade  4,  responses  to  item  H801601  were  recoded  as  follows: 

Not  at  all  or  missing  -  0 
Some  -  1 

A  lot  -  2 


STUD.INV        (How  much  have  you  studied  about  people  who  invented  things  and 
made  new  discoveries?) 

For  grade  4,  responses  to  item  H801701  were  recoded  as  follows: 

Not  at  all  er  missing  -  0 
Some  -  1 

A  lot  -  2 


HIS  TAKE        (Did  you  study  or  expect  to  study  U.S.  history  in  the  following 
grades?) 

For  grade  8,  responses  to  items  H800701  to  H800704  (study  in  fifth, 
sixth,  seventh,  eighth  grade)  were  recoded  as  follows: 

Yes  -  1 

No  -  2 

I  don't  know  -  3 

The  variwiOle  HIS  TAKE  was  then  coded  as  the  sum  of  all  "Yes"  responses, 
ranging  from  0  to  4. 
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EARLY  H         (How  much  have  you  studied  these  periods  of  U.S.  history?) 

For  grade  8,  responses  to  items  H801801  to  H8OI8O0  were  recoded  as 
follows: 

A  lot  -  2 

Some  -  1 

-Not  at  all    -  0 

The  average  of  the  six  recoded  variables  formed  the  variable  EARLY  H,  whose 
values  riviged'- from  0  to  2. 

POST  CH         (How  much  have  you  studied  these  periods  of  U.S.  history?) 

For  grade  8,  responses  to  items  H801807  to  H801811  were  recoded  as 
follows: 

A  lot  -  2 

Some    .  -  1 

Not  at  all    -  0 

The  average  of  the  five  recoded  variables  formed  the  variable  POST  CH,  whose 
values  ranged  from  0  to  2. 

TEACH  H         (How  often  does  your  social  studies  teacher  ask  you  to  do  these 
things?) 

For  grade  8,  re.sponses  tc  items  H802501  to  H802511  were  recoded  as 
follows: 

Almost  every  day  -  4 

Once  or  twice  a  week  -  3 
Once  or  twice  a  month  -  2 
A  few  times  a  year  «  1 

Never  -  0 

The  average  of  the  11  recoded  variables  formed  the  variable  TEACH  H,  whose 
values  ranged  from  0  to  A. 

TEXTBOOK        (How  much  difficulty  do  you  have  reading  your  social  studies 
textbooks?) 

For  grade  8,  responses  to  item  S008801  were  recoded  as  fellows: 

A  lot  -  2 

Some  -  1 

None  -  0 
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HIS  TAKE       (Did  you  study  or  expect  to  study  U.S.  history  in  the  following 
grades?) 

For  grade  12,  rt:,sponses  to  items  H800801  to  H800804  (ninth,  tenth, 

eleventh,  twelfth  grade)  were  recoded  as  follows: 

Yes  -  1 

No  -  2 

I  don' t  know    -  3 

The  variable  HIS  TAK?  was  then  coded  as  the  suin  of  all  "Yes"  responses, 
ranging*  from  0  to  4 . 

HIS  COUR       (Since  the  beg:.nning  of  ninth  grade,  how  much  U.S.  history 
coursework  h^ve  you  completed  up  to  now?) 

For  grade  12,  responses  to  item  H800901  were  recoded  as  follows: 


None  -  00 

Less  than  half  a  year  -  25 

Half  a  year  -  50 

Between  half  a  year  and  one  year  -  75 

One  year  -  100 

More  than  one  year  -  200 


AP  HIST         (Have  you  taken  or  are  you  taking  Advanced  Placement  American 
history?) 

For  grade  12,  responses  to  item  H801001  werv  recoded  as  follows: 

Yes  -  1 
No       -  0 


AM  HIST         (Since  the  beginning  of  ninth  grade,  how  much  have  you  studied  the 
following  periods  of  U.S.  history?) 

For  grade  12,  responses  to  items  H801901-H801909  and  H801913  were 
recoded  as  follows: 

A  lot  -  2 

Some  -  1 

Not  at  all    -  0 

The  average  of  the  10  recoded  variables  formed  the  variable  AM  HIST,  whose 
values  ranged  from  0  to  2. 
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UORLD  H         (Since  the  beginning  of  ninth  grade,  how  much  have  you  studied  the 
following  periods  of  U.S*  history?) 

For  grade  12,  responses  to  items  H801910-H801912  were  recede i  as 
follows : 

A  lot  -  2 

Some  -  1 

Not  at  all    -  0 

The  average  of  the  three  recoded  variables  formed  the  variable  WORLD  H,  whose 
values  ranged  from  0  to  2. 

MIN/WOM         (To  what  extent  have  you  studied  the  history  and  contributions  of 
minorities  and  women?) 

For  grade  12,  responses  to  item  H502001  were  recoded  as  follows: 

A  lot  -  2 

Some  -  1 

Not  at  all    -  0 


LIKE  H  (How  much  do  you  likr     "udying  United  States  history?) 

For  grade  12,  responses  to  item  H802101  were  recoded  as  follows: 

It  Is  one  of  my  favorite  subjects  «  3 

It  is  interesting  «  2 

I  like  several  other  subjects  better  -  1 

I  have  never  studied  United  States  history  -  0 

HAD  HIST        (Have  you  ever  had  a  United  States  history  class?) 

For  grade  12,  responses  to  item  H802201  were  recoded  as  follows: 

Yes  -  1 
No       -  0 
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HOMEWK  H       (Hew  cuch  time  have  you  usually  spent  each  week  on  homework  for 
your  U,S.  history  class?) 


For  grade  12,  responses  to  item  H802301  werp  receded  as  follows: 


I  usually  haven't  had  homework  assigned  -  0 

I  have  had  homework,  but  I  usually  haven't  done  it  -  0 

Less  than  1  hour  -  ,5 

1  hour  -  1 

2  hours  -  2 

3  hours  -  3 

4  hours  -  4 

5  hours  or  more  -  5 


GRADES  H       (What  kind  of  grades  have  you  go::ten  in  your  U,  S,  history  class?) 
For  grade  12,  responses  to  item  H80.?401  were  receded  as  follows: 


Mostly  A 

4.0 

About  half  A  and  half  3 

3.5 

Mostly  B 

3.0 

About  half  B  and  half  C 

2.5 

Mostly  C 

2.0 

About  half  C  and  half  D 

1.5 

Mostly  D 

1.0 

Mostly  below  D 

mm 

0.0 

TEACH/H         (How  often  has  your  U.S.  history  teacher  asked  you  to  do  the 
following  things  in  class?) 

For  grade  12,  responses  to  items  H802601-H802611  were  receded  as 
follows: 


Almost  every  day  -  4 

Once  or  twice  a  week  -  3 

Once  or  twice  a  motith  -  2 

A  few  times  a  year  -  1 

Never  -  0 


The  average  cf  the  11  receded  *>riables  formed  the  variable  TEACH/H,  whose 
values  ranged  from  0  to  4. 
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TEXTBOOK       (How  much  difficulty  h&ve  you  had  reading  your  U.  S.  history 
textbook?) 

For  grade  12,  responses  to  item  H802801  were  recoded  as  follows: 

A  lot  -  2 

Some  -  1 

None  -  0 


Geography  Cross -sectional  Variables    (Grade  12) 

GEO  TAKE       (Did  you  take  or  do  you  expect  to  take  a  geography  course  in  the 
following  grades?) 

Responses  to  Itcims  G8003G1-G800304  (ninth,  tenth,  eleventh,  twelfth 
grades)  were  recoded  as  follows: 

Yes  -  1 

No  -  2 

I  don't  know  -  3 

The  variable  GEO  TARE  was  then  coded  as  the  sum  of  all  "Yes"  responses, 
ranging  from  0  to  4, 

PHYSICAL       (How  much  have  you  studied  the  following  topics  in  geography?) 

Responses  to  items  G800201-G800209  (physical  geography  topic?0  were 
recoded  as  follows: 


A  lot  -  3 

Some  -  2 

Very  little  -  1 
Not  at  all    -  0 


The  average  of  the  nine  recoded  vairiables  formed  tb  ^  variable  PHYSICAL,  who«:e 
values  ranged  from  0  to  3. 

SOCIAL  (How  much  have  you  studied  the  following  topics  in  geography?) 

Responses  to  items  G800210-G800215  (social  geography  topics)  were 
r    oded  as  follows: 


A  lot  -  3 

Some  -  2 

Very  little  -  1 
Not  at  all    -  0 
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The  average  of  the  five  receded  variables  formed  the  variable  SOCIAL,  whose 
values  ranged  from  0  to  3. 


The  following  nine  variables  were  created  from  responses  to  nine  items 
that  stemmed  from  the  question  "Since  the  beginning  of  ninth  grade,  how  much 
of  the  following  geography  coursework  have  you  completed  up  to  now?" 

W  GEO  (World  geography  coursework  taken) 

W  HIS/G  (World  history  and  geography  coursework  taken) 

STAT/REG  (State/regional  geography  coursework  taken) 

US  GEO  (United  States  geography  coursework  taken) 

US  H/G  (United  States  history  and  geography  coursework  taken) 

PHY/GEO  (Physical  geography /earth  science  coursework  taken) 

ECON/P  (Economic  and  political  geography  coursework  taken) 

HUM/CULT  (Human  and  cultural  geography  coursework  taken) 

URBAN  G  (Urban  geography  coursework  taken) 

These  variables  were  created,  respectively,  from  items  G800101-G800109 . 
Responses  to  these  items  were  first  recoded  as: 

Yes,  in  grade  9  -  1 

Yes,  in  grade  10  -  1 

Yes,  in  grade  11  -  1 

Yes,  In  grade  12  -  1 

No,  I  haven't  taken  -  0 

To  form  the  new  variables,  the  recoded  responses  were  then  collapsed  as 
follows: 

Any  "Yes"  -  1 

No,  I  haven't  taken  -  0 
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Mathematics  Trend  Variables 


Observed  Race  x  Observed  Sex 

This  variable  is  defined  as  the  following  combinations  of  the  variables 
RACE  and  SEX: 


SAGE 

SEX 

1 

White,  American 
or  Unclassified 

Indian, 

Other , 

Male 

2 

Black 

Male 

3 

Hispanic 

Male 

4 

Asian  American 

Male 

5 

White,  American 
cr  Unclassified 

Indian, 

Other , 

Female 

6 

Black 

Female 

7 

Hispanic 

Female 

8 

Asian  American 

Female 

Observed  Race  x  Parents'  Education 

This  variable  is  defined  as  the 
RACE  and  PARED: 


RACE 


1 

White,  American  Indian, 
or  Unclassified 

Other, 

2 

White,  American  Indian, 
or  Unclassified 

Other , 

3 

White,  American  Indian, 
or  Unclassified 

Other , 

4 

White,  American  Indian, 
or  Unclassified 

Other, 

5 

White,  American  Indian, 
or  Unclassified 

Other, 

6 

Black 

7 

Black 

8 

Black 

9 

Black 

10 

Black 

11 

Hispanic 

12 

Hispanic 

13 

Hispanic 

14 

Hispanic 

15 

Hispanic 

16 

Asian  American 

17 

Asian  American 

18 

Asian  American 

19 

Asian  American 

20 

Asian  American 

ERIC 


following  combinations  of  the  variables 


PARED 


Less  than  high  school  education 

High  school  graduate 

Some  education  after  high  school 

College  graduate 

Missing 

Less  than  high  school  education 

High  school  graduate 

Some  education  after  high  school 

College  graduate 

Hissing 

Less  then  high  school  education 

High  school  graduate 

Some  education  after  high  school 

College  graduate 

Missing 

Less  than  high  school  education 

High  school  graduate 

Some  education  after  high  school 

College  graduate 

Missing 
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Observed  Race  x  Language  in  the  Home 


Items  B003201  (age  9)  and  B003301  (ages  13  and  17)  asked  students  how 
often  people  in  their  home  speak  a  language  other  than  English.  This 
conditioning  variable  is  defined  by  the  following  combinations  of  the 
variables  RACE  and  B003201  (for  age  9)  or  RACE  and  B003301  (ages  13  and  17) 

Age  9 

RACE  B003201 


1 

White,  American  Indian, 
or  Unclassified 

Other, 

Always 

2 

White,  American  Indian, 
or  Unclassified 

Other, 

Sometimes 

3 

White,  American  Indian, 
or  Unclassified 

Other 

Never 

4 

Black 

Always 

5 

Black 

Sometimes 

6 

Black 

Never 

7 

Hispanic 

Always 

8 

Hispanic 

Sometimes 

9 

Hispanic 

Never 

10 

Asian  American 

Always 

11 

Asian  American 

Sometimes 

12 

Asian  American 

Never 

Ages  13  and  17 

RACE  B003301 


1 

White,  American  Indian, 
or  Unclassified 

Other, 

Always  or  mostly 

2 

White,  Aiiierican  Indian, 

Other, 

or  Unclassified 

Occasionally  or  half 

the 

time 

3 

White,  American  Indian, 
or  Unclassified 

Other, 

Never 

4 

Black 

Always  or  mostly 

5 

Black 

Occasionally  or  half 

the 

time 

6 

Black 

Never 

7 

Hispanic 

Always  or  mostly 

8 

Hispanic 

Occasionally  or  half 

the 

time 

9 

Hispanic 

Never 

10 

Asian  American 

Always  or  mostly 

11 

Asian  American 

Occasionally  or  half 

the 

time 

12 

Asian  American 

Never 
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Appendix  E 


REVISION  OF  1984  NAEP  POSTSTRATIFICATION  WEIGHTS 
FOR  GRADE  A/AGE  9  AND  GRADE  8/AGE  13 


Keith  F.  Rust 
Westat,  Inc* 


A  comparison  of  the  propor*    ms  of  9-ysar-old  students  who  were  in 
grade  4,  based  on  weighted  data,  revealed  an  inconsistency  betweer*  the  1984 
main  sample  results  and  those  for  bridge  studies  in  subsequent  years.  In 
1984,  the  percentage  of  9 -year-old  students  in  grade  4  was  74.9.     For  three 
subsequent  bridges,  the  percentage  ranged  from  62.6  to  66.1. 

A  consideration  of  the  method  of  obtaining  the  separate 
poststratif ication  f«actors  for  those  students  both  grade  and  age  eligible, 
those  eligible  by  age.  alone,  and  those  eligible  by  ^rade  alone,  used  in  1984 
but  not  for  subsequent  bridges,  revealed  the  possibility  of  improving  the 
approach  used  to  derive  the  independent  estimates  which  constitute  the  major 
component  of  the  numerators  of  each  poststratif ication  factor.  This 
improvement  pertained  to  the  poststratif ication  procedure  for  grade  4/age  9 
and  grade  8/age  13,  but  not  grade  11/age  17. 

The  possibility  of  improvement  arose  because  the  independent  estimates 
were  derived  using  Current  Population  Survey  data  on  the  distribution  over 
grades  of  the  population  by  whole  years  of  age.    These  ages  are  as  of  early 
October,  the  time  each  year  the  Current  Population  Survey  in  which  this 
information  is  collected  is  conducted.     The  age  definition  for  ages  9  and  13 
used  in  1984  means  that  this  distribution  is  required  as  of  January  1.  (For 
age  17,  and  for  all  three  ages  for  the  main  samples  in  1986,  the  appropriate 
date  is  October  1,  consistent  with  the  Current  Population  Survey  data, 
consequently,  the  problem  of  inconsirtency  between  NAEP  and  the  Curt  ant 
Population  Survey  did  not  occur  for  these  cases.) 

Evidence  from  the  1984  and  1988  NAEP  samples  shows  clearly  that  the 
proportion  of  9-/ear-olds  who  were  in  grade  4  and  13-year-olds  who  were  in 
grade  3  declined  between  October  1  and  the  following  January  1.     That  is, 
there  were  mcr:*  fourth  graders  who  had  their  tenth  birthday  during  this  period 
than  there  were  fourth  graders  who  had  their  ninth  birthday.    The  difference 
was  sufficiently  great  as  to  decrease  the  percentage  of  9-year-olds  who  were 
age-eligible  by  about  10  percentage  points.    A  similar  but  less  marked 
decrease  also  occurred  at  age  13 . 

Independent  estimates  and  the  resulting  poststratif ication  factors  were 
recomputed  in  a  way  that  recognized  this  shift.    The  magnitude  in  the  shift 
was  estimated  from  NAEF  data,  this  being  the  only  source  of  information 
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available.  We  note  that  the  shift  proved  very  consistent  between  the  1984  and 
1988  samples,  when  the  same  age  and  grade  definitions  were  used. 

The  1988  poststratif ication  procedure,  which  differed  from  that  used  In 
1984  and  1986  in  a  number  of  ways,  was  performed  in  a  manner  that  also 
accounted  for  this  shift  in  the  grade/age  distribution.    Hence,  no  revision  of 
the  1988  poststratification  factors  is  required. 
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Appendix  F 
1988  NAEP  IRT  PARAMETERS 


Appendix  F  contains  13  tables  of  IRT  (Item  response  theory)  parameters 
for  NAEP  Items  that  were  used  In  each  subject  area  and  study  (cross -sectional 
or  trend)  for  which  IRT  scales  were  created. 

For  each  NAEP  Item  used  In  scaling,  the  tables  show  the  corresponding 
IRT  parameters  (A,  B,  and  C)  and  standard  errors  (S.E.)»  the  block  in  which 
the  Item  appears  for  each  age  class  (BLOCK),  and  the  order  in  which  the  item 
appears  within  the  block  (ITEM). 

IRT  parameters  for  items  used  in  cross -sectional  scales  are  shown  for 
reading  in  Table  F-1,  for  civics  in  Table  F-3,  for  U.S.  history  in  Table  F-6, 
and  for  gc^ography  in  Table  F-7.     IRT  parameters  for  items  used  in  trend  scales 
are  shown  for  reading  in  Table  F-2,  for  civics  in  Tables  F-4  and  F-5,  for 
mathematics  in  Tables  F-8  to  F-10,  and  for  science  in  Tables  F-11  to  F-13. 

Note  that  item  parameters  shown  in  this  appendix  are  in  the  metrics  used 
for  the  original  calibration  of  the  scale.    The  tr'^.nsformations  needed  to 
represent  these  parameters  in  terms  of  the  metric  of  the  final  reporting 
scales  are  given  in  Chapters  10  through  l"". 
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Table  F-1 


1988  irT  Parameters,  Reading  Cross-secclonal 
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Table  F-1  (continued) 


1988  IRT  Parameters,  Reading  Cross -sectional 
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(0. 029) 

0.49Z 

(0.026) 

0.283 

n.oil) 



_ 

R5 

3 

nwxowws 

X  .  X^o 

/O  O'lO) 

0.139 

( 0 . 022 ) 

0.240 

014) 

R3 

4 

nv XDwwS 

9   1  R7 
£  .  XD  / 

0  940 

(0 .029) 

0.247 

on) 

R3 

5 

n wXD w wO 

1.295 

(n  097^ 

0.489 

( 0 . 024 ) 

0.252 

(u . Oil) 

_  ^ 

R3 

6 

nu^i iwx 

0  7S7 

I U  .  U  X/  / 

-1. 807 

(0. 046) 

0.206 

(0. 016) 

R7 



— 

— 

nu^x  xu^ 

0  9A3 

(0. 022) 

-0.997 

(0.031) 

0. 166 

(0. Oil) 

R7 

5 



— 

— 

nv^x  xuo 

1. 096 

(0. 025) 

-1. 082 

(0.033) 

0.207 

(0. Oil) 

R7 

5 

— 

— 

X  ^  w  X 

1.337 

(0. 050) 

-0.297 

( 0 . 026 ) 

0.208 

(0.008) 

R7 

12 



_  _ 

— 

nu^x^  w£ 

2.202 

(0. 074) 

-0»354 

(0.032) 

0.214 

(0. 007) 

R7 

13 

_  _ 

— 

— 

tt  W  ^  X  *f  W 

1 . 47-5 

(0.032) 

-1. 130 

(0.038) 

0.282 

(0.012) 

R? 

14 



_  _ 

— 

— 

Hn91 404 
nw£  X  ^ 

2.006 

(0. 113) 

0. 081 

(0.032) 

0.205 

(0.006) 

R7 

15 



_  _ 

-- 

— 

SOI 
n w£  X jux 

0.859 

(0. 025) 

-0.780 

(0.03X) 

0.246 

(0.010) 

R5 

1 



-- 

-- 

Zi  w^X  Jw£ 

l.llE' 

-1. 607 

(0.037) 

0.201 

(0. 014 ) 

2 

a.— 

_  _ 

— 

— 

llw^X^wO 

X  .  XX  X 

/o  01Q^ 

-1. 975 

(0.043) 

0.223 

( 0 . 0 18 ) 

3 



_  _ 

— 

— 

unoi  <cnx 

/n  n')0> 

^  U  ,  \JO\f  } 

-0  70R 

tn  09Q) 

0  94A 

tn  OOQ) 

R5 

— 

— 

1.582 

tn  on^ 

-0.837 

(0.030) 

0. 198 

(0.009) 

R5 

5 



— 

— 

nu^ic'iux 

0  7A1 

u .  /  ox 

tn  09R) 

-0 . 009 

tn  091) 

V  U . w£X } 

0  291 

(0.013) 

R4 

15 

nv^XOUA 

0.70S 

tn  01Q) 

1. 098 

(0.035) 

0. 188 

(0.009) 

_  _ 

R4 

16 

flUZxDwO 

0  RH 

I U . Uw£ / 

2.241 

tn  1 1Q) 

i  U  .  X  Xo  } 

0.38X 

(0.007) 

R4 

17 

1.488 

(0.033) 

0.393 

/ 0 . 026 ) 

0.278 

(0.011) 

__ 



R4 

18 

Hn91 ROS 

1.242 

(0. 023) 

0.869 

(0.029) 

0.276 

(0.009) 



R4 

19 

ROOOlOl 

S\V  W  W  A  W  X 

1.095 

(0.024) 

-0.970 

(0.030) 

0. 176 

(0.010) 

R5 

6 



_  ^ 

— 

— 

Aw w wXw£ 

1.391 

(0. 064) 

-0.035 

(0.024) 

0.200 

(o.ooy) 

n5 

7 

— 

R000103 

1.702 

(0.043) 

-0.699 

(0.  '*32) 

0.276 

(0.009) 

R5 

8 

_  « 

— 

— 

R000104 

1.766 

(0.031) 

-1.362 

(0.041) 

0.313 

(0.012) 

R5 

g 

_  _ 



— 

— 

R000201 

0.983 

(0. 030) 

-0.839 

(0.034 ) 

0.304 

(0.010) 

R5 

10 





— 

— 

on 00202 

2.351 

(0.044) 

-0.873 

(0.037) 

0.252 

(0.009) 

R5 

ii 



— 

— 

TJ000203 

1.462 

(0.027) 

-1.363 

(0.039) 

0.273 

(0.013) 

R5 

12 

— 

Q000204 

2.011 

(0.035) 

-1. 165 

(0.040) 

0.273 

(0.011) 

R5 

13 



_  _ 

— 

— 

R000205 

1.389 

(0.027) 

-1.305 

(0.038) 

0.255 

(0.013) 

R5 

14 



__ 

— 

R000206 

1.068 

(0.039) 

0.130 

(0.021) 

0.000 

(0.000) 

R5 

15A 

«- 



-- 

R000301 

1.510 

(0.053) 

-0.355 

(d.028) 

C  .257 

(0.G08) 

R6 

1 

__ 

-- 

— 

R000302 

1. 134 

(0.027) 

-0.862 

(0.029) 

0.191 

(0.010) 

R6 

2 

„  _ 

— 

— 

R000303 

1.076 

(0. 034) 

-0.567 

(0.029) 

0.258 

(0.009) 

R6 

3 



-- 

R000304 

1.035 

(0.027) 

-0.895 

(0.032) 

0.241 

(0.010) 

RB 

A 



- 

— 

R000401 

1.504 

(0.056) 

-0.260 

(0.026) 

0.2i7 

(0.007) 

R6 

5 



— 

0.848 

(0. 017) 

-1.994 

(0.047) 

0.222 

(0.017) 

R6 

6 

_  _ 

— 

000040*) 

1. 052 

(0.054) 

0. 112 

( 0 . 025 ) 

0.238 

(0.007) 

R6 

7 

_  ^ 

_  _ 

— 

— 

P000404 

1  336 

tn  025^ 

-1. 258 

(0.034 ) 

0.186 

(0.012) 

R6 

8 

_  _ 

ponnsoi 
nuuwjwx 

1.583 

(0.030) 

""i.  197 

(0. 036) 

0.236 

(0.011) 

R6 

9 



_  _ 

R000502 

1.656 

(0.060) 

-0.395 

(0.030) 

C.270 

(0.008) 

R6 

10 

__ 

R00P503 

1.194 

(0.067) 

0.449 

(0.037) 

u.162 

(0.^06) 

R6 

11 

R000504 

0.726 

(0.027) 

-0.673 

(0.032) 

0.261 

(0.010) 

•»* 

12 

R00050S 

1.435 

(0.054) 

-0.397 

(0.030) 

0.245 

(0.008) 

Rb 

13 

R000601 

1.079 

(0.022) 

-1.326 

(0.03.^) 

0.197 

(0.012) 

R7 

1 

R000602 

1.029 

(0.028) 

-0.829 

(0.031) 

0.240 

(0.010) 

R7 

2 

R000603 

1.354 

(0.030) 

-0.973 

(0.032) 

0.231 

(0.010) 

R7 

3 

R000701 

1.338 

(0.056) 

-0.054 

(0.023) 

0.171 

(0.007) 

R7 

7 

R000702 

1.767 

(0.076) 

-0.152 

(0.027) 

0.216 

(0.007) 

R7 

8 

^000703 

1.472 

(0.030) 

-1.083 

(0.035) 

0.257 

(0.011) 

R7 

9 

R000704 

0.556 

(0.017) 

-1.821 

(0.058) 

0.221 

(0.015) 

R7 

10 

R000705 

0.661 

(0.026) 

0.537 

(0.030) 

0.273 

(0.010) 

R7 

11 

R000801 

0.7*1 

(0.015) 

-1.541 

(0.036) 

0.231 

(0.012) 

R8 

6 

R8 

10 

R000802 

1.206 

(0.019) 

-1.311 

(0.028) 

0.195 

(0.011) 

R8 

7 

R8 

11 

R000803 

0.941 

(0.022) 

-0.452 

(0.017) 

0.164 

(O.OO'O 

R8 

8 

R8 

12 

R00080A 

1.631 

(0.035) 

-0.402 

(0.019) 

0.139 

(0.0C6) 

R8 

9 

R8 

i3 

R000805 

1.462 

(0.031) 

-0.702 

(0.024) 

0.226 

(0.008) 

R8 

10 

R8 

14 

6/", 

ERIC 


Table  F-1  (continued) 


1988  IRT  Parameters,  Reading  Cross -sectional 


KAEP  ID 

S.E. 

B 

S.E. 

C 

R000806 

1.6A7 

(0.045) 

-0.799 

(0.036) 

0.537 

R000807 

0.881 

(0.0)6) 

-1.090 

(0.025) 

0.000 

R000901 

1.531 

(0.02::.) 

-0.797 

(0.021) 

0.20C 

R000902 

1.185 

(0.024) 

-0.808 

(0.023) 

0.297 

R000003 

1.523 

(0.022) 

-1.074 

(0.023) 

0.165 

R000g04 

2.829 

(0.106) 

0.473 

(0.045) 

0.121 

R000904 

1.973 

(0.043) 

0.009 

(0.018) 

0.176 

R00090S 

1.4A8 

(0.037) 

-0.928 

(0.033) 

0.199 

ROOlOOl 

1.172 

(0.043) 

-0.791 

(0.039) 

0.273 

R001002 

1.336 

(0.047) 

-0.166 

(0.023) 

0.250 

R001003 

1.A98 

(0.047) 

0.144 

(0.022) 

0.197 

R001004 

1.499 

(0.052) 

-0  165 

(0.024) 

0.223 

ROOllOl 

1.375 

(0.034) 

0.932 

(0.035) 

0.132 

R001102 

1.797 

(0.046) 

0.729 

(0.039) 

0.351 

R001103 

1.089 

(0.035) 

0.570 

(0.029) 

0.190 

R001201 

1.98  A 

(0.067) 

-0.2n 

(0.026) 

0.178 

R001202 

1.819 

(0.062) 

0/038 

(0.O26) 

0.294 

•  R001203 

1.695 

(0.061) 

-0.356 

(0.0?2^ 

0.197 

R001204 

1.57* 

(0.049) 

-0.123 

(0.022) 

0.159 

R001205 

1.7A9 

(0.059) 

-0.541 

(0.034) 

0.250 

R001301 

1.020 

(0.034) 

-0.080 

(0.019) 

0.189 

R001302 

1.563 

(0.051) 

0.301 

(0.027) 

0.333 

R001303 

2.042 

(0.081) 

-0.289 

(0.032) 

0.322 

R001304 

2.428 

(0.041) 

0.924 

(0.038) 

0.197 

R001305 

1.540 

(0.057) 

-0.457 

(0.031) 

0.282 

R001401 

1.863 

(0.057) 

0.086 

(0.O24) 

0.189 

R001^02 

2.344 

(0.O85) 

-0.007 

(O.ObD) 

0.268 

RC01403 

2.321 

(0.O53) 

0.398 

(O.031) 

0.262 

R001404 

2.271 

(0.091) 

0.090 

(O.031) 

0.308 

R00140& 

0.773 

(0.032) 

0.057 

(0.020) 

0.238 

R001501 

0.809 

(0.021) 

-0.974 

(0.030) 

0.202 

R001S02 

0.61^ 

(0.015) 

0.182 

(0.013) 

0.146 

R001503 

1.190 

(0.033) 

-0.909 

(0.033) 

C.238 

R001504 

1.460 

(0.031) 

-0.032 

(0.017) 

0.261 

R001601 

1.575 

(0.037) 

-0.642 

(0.026) 

0.166 

R001602 

1.406 

(0.037) 

-0.401 

(0.021) 

0.207 

R001603 

1.245 

(0.029) 

-0.097 

(0.016) 

0.240 

R00160A 

0.'  . 

(0.020) 

-0.951 

(0.029) 

0.172 

R001605 

1.039 

(0.019) 

0.463 

(0.016) 

O.207 

R001701 

1.132 

(0.025) 

0.153 

(0.017> 

'  35 

R001702 

1.936 

(0.058) 

-0,498 

(0.028) 

U.261 

R001703 

0.670 

(0.016) 

0.939 

(0.026) 

0.180 

R00170A 

1.057 

(0.019) 

0.568 

(0.018) 

0.224 

R001801 

1.493 

(0.033) 

-0.017 

(0.017) 

0.210 

R001602 

2.274 

(0.031) 

0.365 

(0.019) 

0.149 

R001803 

1.087 

(0.018) 

0.771 

(0.021) 

0.235 

R00180A 

2.053 

(0.035) 

0.308 

(0.021) 

0.294 

R001805 

1.840 

(0.028) 

0.355 

(0.019) 

0.225 

R001806 

0.955 

(0.020) 

0  184 

(0.015) 

0.215 

R001901 

0.692 

(0.031) 

-i.xJ5 

(0.057) 

0.237 

ROD  1902 

1.535 

(0.036) 

0.250 

(0  024) 

0.241 

R001903 

1.172 

(0.027) 

0.255 

(0.022) 

0.226 

R00190A 

1.864 

(0.048) 

0.130 

(0.027) 

0.273 

R001905 

1.307 

(0.039) 

-0.237 

(0.026) 

0.191 

R002O01 

1.762 

(0.057) 

-0.080 

(0.029) 

0.272 

R002002 

1.099 

(0.025) 

0.412 

(0.023) 

0.259 

R002003 

1.463 

(0.O30) 

0.329 

(0.023) 

0.198 

R00200A 

1.179 

(0.032) 

0.115 

(0.023) 

0.285 

R002005 

0.837 

(0.022) 

0.716 

(0.028) 

0.279 

R002101 

0.442 

(0.026) 

2.427 

(0.145) 

0.305 

R002102 

1.160 

(0.024) 

0.363 

(0.021) 

0.169 

R002103 

1.528 

(0.047) 

-0.162 

(0.027) 

0.213 

R00210A 

1.117 

(0.030) 

-0.030 

(0.022) 

0.245 

R002201 

1.324 

(0.027) 

0.951 

(0.034) 

0.417 

GRADE  4/AGE  9        GRADE  8/AGE  13       GRADE  12/AGE  17 


S.E. 

BLOCK 

ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

0.008) 

R8 

llA 

R8 

15A 

— — 

— — 

0.000) 

R8 

IIB 

R8 

15B 

— — 

— — 

0.008) 

R8 

1 

R8 

1 

R8 

1 

0.009) 

R8 

i* 

R8 

2 

R8 

2 

0.009) 

R8 

3 

R8 

3 

R8 

3 

0.004) 

R8 

4 

•— 

— — 

~— 

— ~ 

0.008) 

— — 

— — 

R8 

4 

R8 

4 

0.015) 

R6 

5 

R8 

S 

R6 

5 

0.015) 

R8 

6 

0.011) 

— — 

— — 

Re 

7 

-  — 

-— 

0.009) 

— — 

F.8 

8 

— — 

0.010) 

— — 

— — 

U8 

9 

— — 

— — 

0.006) 

R6 

16 

— 

0.007) 

— — 

R6 

17 

— — 

— — 

0.008) 

— — 

— — 

R6 

18 

— — 

— — 

0.010) 

— - 

— — 

R6 

6 

— — 

— ~ 

0.009) 

— — 

R6 

7 

— — 

— - 

0.011) 

— 

— " 

R6 

8 

-- 

— 

0.010) 

— ~ 

R6 

9 

— - 

-— 

0.013) 

— - 

— ~ 

R6 

10 

0.010) 

— — 

— — 

R6 

1 

— — 

— — 

0.008) 

— - 

— - 

R6 

2 

— — 

0.011) 

— — 

— — 

R6 

3 

— - 

0.005) 

— — 

— — 

R6 

4 

— — 

— — 

0.012) 

— — 

— — 

R6 

5 

— - 

— — 

O.C09) 

— — 

R6 

11 

— — 

0.009) 

R6 

12 

0.006) 

R6 

13 

— 

0.009) 

R6 

14 

0.010) 

— — 

R6 

15 

— — 

0.013) 

R7 

1 

R7 

1 

0.008) 

R7 

? 

R7 

2 

0.014) 

R7 

U 

R7 

0.006) 

R7 

4 

r7 

4 

0.012) 

R7 

5 

R7 

5 

0.010) 

R7 

6 

iJ 

6 

0. 009) 

R7 

7 

R7 

7 

0. 013) 

R7 

8 

R7 

8 

0 .006) 

R7 

9 

R7 

9 

0 .008) 

R7 

10 

R7 

10 

0 .011) 

R7 

11 

R7 

11 

0.006) 

R7 

i2 

K7 

12 

0.006) 

R7 

13 

R7 

IJ 

0.006) 

R7 

14 

R7 

14 

0.006) 

R7 

15 

R7 

15 

0.006) 

R7 

16 

R7 

16 

0 . 007 } 

R7 

17 

R7 

17 

0 . 007) 

R7 

1  o 

io 

R7 

0.006) 

R7 

19 

R7 

19 

0.022) 

R4 

10 

0.012) 

R4 

11 

0.012) 

R4 

12 

0.013) 

R4 

13 

0.017) 

R4 

14 

0.015) 

R4 

1 

0.011) 

R4 

2 

0.011. 

R4 

3 

0.013) 

R4 

4 

0.010) 

R4 

5 

0.007) 

R4 

6 

0.011) 

R4 

7 

0.016) 

R4 

8 

0.014) 

R4 

9 

0.008) 

R6 

1 

649 

683 


Table  F-1  (continued) 


1988  IRT  Parameters,  Reading  Cross-sectional 


WASP  IP 

E002202 
R002203 
R002204 
R00220S 
R002301 
R002302 
2002303 
R002304 
H002401 
R002402 
R002403 
R002404 
R00240S 
R002406 
R002S01 
R002S02 
R002S03 
R002601 
R002602 
R002603 
R002604 
R00260S 
R002701 
R002702 
RO&2703 
R002704 
R00270S 


GI^E  4/AGE  9 
BLOCK  ITEM 


GRADB  8/AGE  13 
BLCQt 


GRADE  12/AGE  17 
BLOCK  ITEH 


1.123 
1.107 
0.M5 
1.076 
2.070 
0.683 
1.982 
2.066 
1.261 
0.777 
1.383 
1.427 
1.482 
0.806 
1.564^ 
1.150 
1.267 
1.442 
1.402 
1.539 
1.338 
1.676 
1.462 
1.327 
2.013 
0.971 
1.940 


(0.051) 

-0.911 

(0.053) 

0.234 

(0.024) 

R6 

2 

(0.023) 

0.535 

(0.023) 

0.238 

(0.010) 

R6 

3 

(0.020) 

1.279 

(0.039) 

0.227 

(0.008) 

R6 

4 

(0.043) 

2.336 

(0.111) 

0.319 

(0.006) 

R6 

5 

(0.033) 

0.569 

(0.027) 

0.244 

(0.009) 

R6 

6 

(0.020) 

0.839 

(0.031) 

0.232 

(0.010) 

R6 

7 

(0.030) 

0.526 

(0.025) 

0.178 

(0.009) 

R6 

8 

(0.036) 

0.411 

(0.026) 

0.203 

(0.010) 

R6 

9 

(0.057) 

-0.479 

(0.037) 

0.287 

(0.019) 

R6 

10 

(0.026) 

1.820 

(0.069) 

0.335 

(0.007) 

R6 

11 

(0.044) 

-0.054 

(0.027) 

0.317 

(0.015) 

R6 

12 

(0.021) 

1.073 

(0.028) 

0.197 

(0.008) 

R6 

13 

(0.029) 

0.550 

(0.026) 

0.262 

(0.010) 

R6 

14 

(0.017) 

1.011 

(0.027) 

0.000 

(0.000) 

R6 

ISA 

(0.026) 

0.547 

(0.024) 

0.227 

(0.010) 

R8 

6 

(0.026) 
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1988  IRT  Parameters,  Reading  Bridge  to  1984 
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Table  F-2  (continued) 


1988  IRT  Parameters,  Reading  Bridge  to  1984 
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1  177 
X  .  X/  / 

(0. 163) 

-ft  ftQI 

fft  0A71 

\ u . uo/ ^ 

ft   1  7A 

u .  x/o 

fO  mn\ 

V 

1 1 
XJ 

V00QO01 
nuuouuw 

ft  A44 

fO  9ft11 

ft  7^^ 

fft  9X91 

n  99R 

fft  m9\ 
(U. UJZ 1 

V 

1  A 
1% 

1(009004 

1.768 

(0.225) 

-0.350 

(0 .109) 

0.240 

(0.02**) 

15 

KOOOlOl 

1.007 

(0.120) 

-1.451 

(0.210) 

0.256 

(0.076; 

K 

16 

H009Z01 

1.795 

(0.172) 

-1.377 

(0.216) 

0.301 

(0.054) 

K 

17 

V009401 

1.882 

(0.127) 

-1.402 

(0.172) 

C.10.N 

(0.036) 

L 

23 

H009601 

1.360 

(0.106) 

-1.872 

(0.207) 

0  133 

(0.053) 

L 

21 

R009701 

1.082 

(0.124) 

-0.854 

(0.112) 

0.164 

(O.O^n 

H 

5 

R009702 

1.959 

(0.227) 

-0.533 

(0.131^ 

0.249 

(0.028) 

H 

6 

R00970? 

1.449 

(0.211) 

-0.165 

(0.097) 

0.258 

'0.029) 

H 

7 

R009704 

1.150 

(0.185) 

0.033 

(0.0C6) 

e.209 

(0.031) 

M 

8 

K00970S 

1.957 

(0.207) 

-0.702 

(0.147) 

0.211 

(0.029) 

M 

9 

R009801 

1.396 

(0.134) 

-2.227 

(0.286) 

0.259 

(0.086) 

N 

12 

R0099O1 

0.976 

(0.117) 

-1.049 

(0.160) 

0.206 

(0.059) 

N 

13 

R010002 

1.290 

(0  137) 

-1.094 

(0.165) 

0.172 

(0.047) 

N 

18 

R010003 

1.657 

«..a94) 

-0.940 

(0.179) 

0.241 

(0.041'.) 

N 

19 

R010102 

1.124 

(0.193) 

-0.050 

(0.111) 

0.267 

(0.0-/) 

N 

21 

R010103 

1.795 

(0.200) 

-1.075 

(0.207) 

0.209 

(0.042) 

N 

22 

R010201 

243 

(O.lJtl) 

-1.932 

(0.245) 

0.244 

(0.078) 

0 

16 

R010301 

.702 

(0.085) 

-2.383 

(0.318) 

0.248 

(0.093) 

0 

15 
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Table  F-2  (continued) 


1988  IRT  Parameters,  Reading  Bridge  to  1984 


GRADE  4 /AGE  9         GRADE  8/AGE  13       GRADE  11/AGE  17 

ITEM 


HAEP  ID 

A 

S.E, 

B 

S.E. 

C 

S.E. 

BLOCK 

ITEM 

BLOCK  ITEM 

BLOCK 

H0104')l 

0.715 

(0.087) 

-1.487 

(0.209) 

0.219 

(0.077) 

0 

20 

-- 

N010402 

0.928 

(0.171) 

0.132 

(0.113) 

0.222 

(0.037) 

0 

21 

— 

-- 

N010403 

1.031 

(0.137) 

0.465 

(0.153) 

0.190 

(0.027) 

0 

22 

— 

K010801 

1.084 

(0.119) 

-0.471 

(0.087) 

0.260 

(0.035) 

Q 

16 

— 

— 

noi0902 

1.564 

(0.153) 

-0.467 

(0.087) 

0.241 

(0.026) 

Q 

18 

— 

N010903 

1.850 

(0.157) 

-0.564 

(0.096) 

0.193 

(0.022) 

Q 

19 

— 

— 

N010904 

1.522 

(0.170) 

-0.245 

(0.080) 

0.275 

(0.024) 

Q 

20 

-- 

HOltQOl 

1.279 

(0.116) 

-0.079 

(0.113) 

0.228 

(0,037) 

R 

5 

Noriooz 

1.657 

(0.167) 

-0.315 

(0.081) 

0.252 

(0.022) 

R 

6 

— 

— 

HOI 1003 

2.416 

(0.169) 

-0.928 

(0.147) 

0,*.41 

(0.024) 

R 

7 

— 

N011004 

1.788 

(0.159) 

-0.543 

(0.097) 

0.226 

(0.023) 

R 

8 

— 

KOlllOl 

1.568 

(0.141) 

-0.541 

(0.090) 

0.197 

(0.025) 

R 

9 

— 

— 

N011201 

0.911 

(0.117) 

-0.259 

(0.085) 

?.260 

(0.037) 

R 

10 

—  — 

N011301 

1.653 

(0.14?) 

-0.756 

(0.111) 

C.211 

(0.028) 

R 

11 

-- 

— 

H011302 

0.992 

(0.119) 

-0.430 

(0.089) 

0.227 

(0.039) 

R 

12 

-- 

— 

H011401 

0.838 

(0.194) 

0.697 

(0.227) 

0.334 

(0.030) 

R 

13 

-- 

— 

N011402 

0.822 

(0.139) 

0.010 

(0.102) 

0.288 

(0.041) 

R 

14 

-- 

— 

N011403 

0.971 

(0.192) 

0.621 

(0.186) 

0.270 

(0.025) 

R 

15 

— 

H01U04 

1.327 

(0.215) 

0.492 

(0.151) 

0.220 

(0.019) 

R 

16 

— 

— 

H013201 

1.655 

(0.211) 

-0.693 

(0.160) 

0.181 

(0.037) 

V 

29 

— 

— 

N013301 

1.232 

(0.161) 

-1.557 

(0.268) 

0.253 

(0.077) 

V 

30 

— 

— 

H013401 

1.203 

(0.177) 

-0.250 

(0.107) 

0.1.S7 

(0,035) 

V 

31 

— 

— 

K013402 

1.438 

(0.189) 

-0.862 

(0.175) 

0.2Q5 

(0.048) 

V 

32 

— 

H033'>03 

1.494 

(0.223) 

-0.278 

(0.116) 

0.199 

(0.033) 

V 

33 

— 

NO:  .Ji 

1.238 

(0.153) 

-0.857 

(0.149) 

0.249 

(0,048) 

M 

13 

— 

— *■ 

N0141O1 

0.758 

(0.071) 

-1.283 

(0.142) 

0.169 

(0.059) 

Q 

21 

— 

— — 

N014201 

1.207 

(0.134) 

-1.218 

(0.189) 

0.136 

(0.052) 

V 

34 

— 

— — 

N014301 

1.755 

(0.191) 

-0.820 

(0.158) 

0.190 

(0,035) 

H 

14 

— 

-— 

H014302 

1.074 

(0.136) 

-0.498 

(0.108) 

0.181 

(0,041) 

15 

N014303 

1.721 

(0.187) 

-1.041 

(0.188) 

0.208 

(0.041) 

» 

16 

N014501 

0.432 

(0.065) 

-2.264 

(0.348) 

0.000 

(0,000) 

V 

35A 

-- 

— — 

N014S02 

0.934 

(0.123) 

-2.664 

(0.406) 

0.000 

(0,000) 

V 

35B 

—  —          — — 

N014S03 

0.624 

(0.133) 

-4.120 

(0.903) 

0.000 

(0.000) 

V 

35C 

—  —          — — 

NOISIOI 

0.932 

(0.110) 

0.343 

(0.168) 

(  234 

(0.067) 

— 

— 

R 

HO 15102 

2.533 

(0.236) 

0.548 

(0.207) 

0.216 

(0.030) 

— 

R 

H01S103 

2.401 

(0.200) 

0.660 

(0.197) 

0.219 

(0,028) 

— 

— 

— —          — — 

R 

H01S104 

1.707 

(0.184) 

0.441 

(0.193) 

0.278 

(0.045) 

— 

— —          — — 

R 

H015201 

1.089 

(0.120) 

-0.766 

(0.150) 

0.227 

(0.085) 

H 

H01SS02 

1.273 

(0.126) 

0.189 

(0.140) 

0.209 

(0,057) 

P 

H01SS03 

0.912 

(0.119) 

0,756 

(0.216) 

0,247 

(0.056) 

P 

H01SS04 

1.189 

(0.121) 

0.110 

(0.138) 

0.220 

(0,062) 

P 

HOISSOS 

0.683 

(0.083) 

-0.175 

(0.145) 

0,247 

(0.087) 

P 

H01S901 

1.021 

(0.133) 

0.371 

(0.204) 

0,333 

(0,068) 

Q 

H01S902 

1.380 

(0.165) 

0.726 

(0.234) 

0.317 

(0,043) 

Q 

HO 15903 

1.182 

(0.129) 

1.101 

(0.224) 

0,153 

(0.032) 

Q 

K016001 

1.043 

(0.122) 

0.033 

(0.164) 

0.285 

(0.078) 

0 

H016002 

1.386 

(0.154) 

1.247 

(0.276) 

0.456 

(0,028) 

0 

HO 16003 

0.906 

(0.103) 

0.354 

(0.157) 

0,205 

(0.065) 

0 

H016004 

1.095 

(0.126) 

0.103 

(0.165) 

0.271 

(0.074) 

0 

H01600S 

1.734 

(0.175) 

0.156 

(0.152) 

0.230 

(0.054) 

0 

H016006 

1.357 

(0.137) 

0.424 

(0.161) 

0.203 

(0.049) 

0 

H017001 

1.518 

(0.157) 

0.484 

(0.175) 

0,213 

(0.042) 

H 

HO170O2 

1.935 

(0.138) 

1.100 

(0.193) 

0.196 

(0.022) 

H 

H017003 

1.833 

(0.129) 

1.770 

(0.248) 

0,177 

(0.016) 

H 

17 
18 
19 
20 
26 
16 
17 
18 
19 
14 
15 
16 
15 
16 
17 
18 
19 
20 
7 
8 
9 
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Table  F-3 


1988  IRT  Parameters,  Civics  Cross-sectional 


B 

0. 663 

(0  03A) 

0. 123 

0 .587 

(0.032) 

-0.679 

0.462 

(0.025) 

-1. 126 

ux 

0.282 

(0 .029) 

0.377 

£UUUw vx 

1.324 

(0.029) 

-0.931 

Pflnnf>ni 

1.043 

(0. 044 ) 

-0.597 

f uuu/ux 

1.691 

(0.056) 

0.256 

irvuuoux 

1.709 

(0. 056) 

-0.252 

pnnnAn9 

1.839 

(0.062) 

0. 171 

n  A99 

(0 . 039) 

-0.269 

iruuxuux 

1  n7Q 

(0.050) 

0. 129 

rliUl  xUl 

n  A'^R 

(0 . 028) 

^1. 332 

ITUUX^UX 

0.446 

(0.029) 

-0.514 

X .  X9X 

(0.065) 

0. 130 

pnni  Ll^^ 

0.690 

(0.031) 

-0.376 

1.225 

(0.064 ) 

-0.003 

cUU  X  DVX 

0.86 1 

(0.034) 

-0.791 

pnni 7ni 

I;  UU  X  /  U  X 

0. 743 

(0 .029) 

-0.939 

pnni  Am 

tVv X  oux 

1. 074 

(0.041) 

-0,306 

pnni nn9 

1  AQA 

tn  OAO) 

\V  m  \JO\J  / 

0. 135 

onni on i 
rUUXgUX 

(0.059) 

1 . 142 

pnn^nni 

2.067 

-0.011 

pnn9 1 ni 

Zvv^ XU  X 

2. 981 

tn  089) 

1. 196 

pnnooni 

0 .785 

(0.048) 

"0 .915 

Pfln^'^ni 

1 . 533 

(0.055) 

o!l32 

pon^&ni 

1.557 

(0.064) 

1.680 

P002S01 

1.371 

(0.049) 

1.091 

Pfl07fiQ1 

f           W  W  X 

1.568 

(0.060) 

0.059 

P002701 

0.775 

(0.039) 

0.573 

0.555 

(0.086) 

2.894 

P003201 

1.053 

(0.020) 

0.544 

in'^&ni 

4k  wUi9 1U  X 

1.765 

(0.021) 

1. 159 

P003S01 

1.676 

(0.013) 

1.525 

1.545 

(0 .019) 

0.974 

PQ03701 

1.090 

(0.023) 

0.287 

pnn'^Ani 

f  UUi90U  X 

1.815 

(0.023) 

1.233 

pon'^oni 

cUUi99U  X 

1.523 

(0 .019) 

0.942 

POOiOOl 

1.795 

(0.067) 

-0.068 

PflQ&OQ? 

f  W  U  *9VV^ 

0.447 

(0.020) 

0. 191 

pnn&nn') 

1. 649 

(0.055) 

-0.571 

P004004 

0.547 

(0.022) 

1.263 

POO&OQS 

f  W  W  *f  WUaJ 

1.215 

(0.026) 

1. 491 

P004007 

2.269 

(0.034) 

0.899 

* w www 

1.336 

(0.047) 

-0.367 

?004010 

^  W  V  "f  V  xw 

1.805 

(0.061) 

-0.582 

P004C11 

0.506 

(0.020) 

1.061 

F004012 

0.697 

(0.038) 

2.639 

P004101 

1.270 

(0.033) 

-0.231 

F004201 

1.631 

(0.043) 

-0.266 

F004301 

1.033 

(0.019) 

0.733 

F004^01 

0.703 

(0.028) 

0.037 

F004S01 

2.455 

(0.033) 

1.019 

F004601 

0.782 

(0.029) 

-0.145 

F004701 

0.788 

(0.'^27) 

0.520 

P004801 

1.069 

(0.032) 

-O.OOl 

FOO^QOl 

1.306 

(0.038) 

-0.193 

F0OS301 

1.248 

(0.037) 

0.328 

F0OS4O1 

1.362 

(0.061) 

-0.4aC 

POOSSOl 

0.955 

(0.039) 

-0.644 

FOOSS02 

0.898 

(0.039) 

-1.497 

F005S04 

1.549 

(0.064) 

-1.127 

FOOSSOS 

0.878 

(0.043) 

-0.853 

P00S601 

1.832 

(0.060) 

-0.591 

F005701 

1.032 

(0.027) 

-0.209 

CmADE  4/AGE  9 


S.E. 

C 

S.E. 

BLOCK 

ITEM 

(0.325) 

0.468 

(0.008) 

C4 

1 

(0.044) 

0.485 

(0.010) 

C4 

2 

(0.064) 

0.443 

(0.012) 

C4 

3 

(0.045) 

0.510 

(0.008) 

C4 

4 

(0.029) 

0.251 

(0.012) 

C4 

5 

(0.036) 

0.509 

(0.0';9) 

C4 

6 

(0.027) 

0.313 

(0.006) 

C4 

7 

(0.024) 

0.258 

(0.009) 

C4 

8 

(0.026) 

0.230 

(0.007) 

C4 

9 

(0.026) 

0.504 

(0.008) 

C4 

10 

(0.029) 

0.519 

(0.007) 

C4 

11 

(0.051) 

0.438 

(0.014) 

C4 

12 

(0.039) 

0.496 

(0.009) 

C4 

13 

(0.030) 

0.471 

(0.008) 

C4 

14 

(0.027) 

0.432 

(0.009) 

C4 

15 

(0.029) 

0.500 

(0.008) 

C4 

16 

(0.038) 

0.463 

(0.011) 

C4 

17 

(0.043) 

0.421 

(0.011) 

C4 

18 

(0.025) 

0.424 

(0.009) 

C4 

19 

(0.034) 

0.410 

(0.007) 

C4 

20 

(0.067) 

0.320 

(0.007) 

— 

(0.030) 

0.252 

(0.012) 

— 

— 

(0.101) 

0.434 

(0.008) 

— 

— 

(0.065) 

0.444 

(0.020) 

-- 

— 

(0.026) 

0.209 

(0.012) 

— 

— 

(0.101) 

0.189 

(0.007) 

— 

— 

(0.058) 

0.355 

(0.008) 

— 

— 

(0.C27) 

0.242 

(0.013) 

— 

(0.038) 

0.297 

(0.010) 

— 

— 

(0.4S5) 

0.405 

(0.007) 

— 

— 

(0.019) 

0.195 

(0.007) 

— 

— 

(0.028) 

0.272 

(0.005) 

— 

— 

(0.031) 

0.171 

(0.004) 

— 

— 

(0.024) 

0.230 

(0.006) 

— 

— 

(0.017) 

0.265 

(0.00b) 

— 

— 

(0.031) 

0.271 

(0.005) 

— 

— 

(0.022) 

0.211 

(0,006) 

— 

— 

(0.028) 

0.469 

(0.010) 

— 

— 

(0.020) 

0.474 

(0.008) 

— 

— 

(0.036) 

0.436 

(0.014) 

— 

— 

(0.055) 

0.447 

(0.007) 

— 

— 

(0.048) 

0.523 

(0.005) 

— 

-- 

(0.037) 

0.494 

(0.006) 

— 

- 

(0.028) 

0.428 

(0.012) 

— 

— 

(0.037) 

0.4<i2 

(0.014) 

— 

-- 

(0.047) 

0.451 

(0.007) 

-- 

(0.153) 

0.509 

(0.005) 

(0.019) 

0.264 

(0.010) 

(0.021) 

0.190 

(0.011) 

(0.021) 

0.214 

(0.007) 

(0.022) 

0.545 

(0.009) 

(0.038) 

0.470 

(0.006) 

(0.021) 

0.497 

(0.009) 

(0.030) 

0.567 

(0.007) 

(0.021) 

0.443 

(0.010) 

(0.024) 

0.430 

(0.011) 

(0.029) 

0.500 

(0.009) 

(0.035) 

0.479 

(0.014) 

(0.035) 

0.466 

(0.013) 

(0.073) 

0.451 

(0.022) 

(0.063) 

0.468 

(0.023) 

(0.050) 

0.457 

(0.017) 

(0.034) 

0.'  74 

(0.016) 

(0.018) 

0.220 

(0.011) 

GRADE  8/AGE  13  GRADE  12/AGE  17 
BLOCX    ITEM  BLOCK  ITEM 


C4 

1 

— 

— 

C4 

2 

— 

— 

C4 

3 

— 

— 

C4 

4 

— 

C4 

5 

— 

C4 

6 

— 

C4 

7 

— 

— 

C4 

8 

— 

— 

C4 

9 

— 

— 

C4 

10 

— 

— 

C4 

11 

— 

— 

C4 

12 



— 

C4 

13 

— 

— 

C4 

14 





C4 

15 



— 

C4 

16 





C4 

17 

— 

— 

C4 

18 

— 

C4 

19 

— 

— 

C4 

20 

— 

— 

C4 

21 

— 

— 

C4 

22 

— 

— 

C4 

23 

— 

— 

C4 

24 

— 

C4 

25 

— 

— 

C4 

26 

— 

— 

C4 

27 

— 

— 

C4 

28 

— 

— 

C4 

29 

— 

-- 

C4 

30 

— 

— 

C2 

1 

C2 

1 

C2 

2 

C2 

2 

C2 

3 

C2 

3 

C2 

4 

C2 

4 

C2 

5 

C2 

5 

C2 

6 

C2 

6 

C2 

17 

C2 

17 

C2 

7 

C2 

7 

C2 

8 

C2 

8 

C2 

9 

C2 

9 

C2 

10 

C2 

10 

C2 

11 

C2 

11 

C2 

12 

C2 

12 

C2 

13 

C2 

13 

C2 

14 

C2 

14 

C2 

15 

C2 

15 

C2 

16 

C2 

16 

C2 

18 

C2 

18 

C2 

^.9 

C2 

19 

C2 

26 

C2 

26 

C2 

20 

C2 

20 

C2 

21 

C2 

21 

C2 

22 

C2 

22 

C2 

23 

C2 

23 

C2 

24 

C2 

24 

C2 

25 

C2 

25 

C5 

1 

C5 

1 

C5 

2 

C5 

2 

C5 

5 

C5 

5 

C5 

6 

C5 

6 

C5 

7 

C5 

7 

C5 

8 

C5 

8 

C5 

3 

C5 

3 

C5 

4 

C5 

4 

654 

6^3 


Table  F-3  (continued) 
1938  IRT  Parameters,  Civics  Cross -sectional 

GRADE  4/AGE  9         GRADE  8/AGE  13       GRADE  . 2/AGE  17 


HAPP  Tn 

A 

A 

g>  .  Xi»  t 

Q 

TK\  cry. 

ITEM 

BLOCK 

ITEM 

BLCX^K 

ITEM 

Pfinf  Ani 

& ww^OU 1 

n  AQH 
U  •  Dow 

(0, 028) 

0 . 002 

( 0 , 022 ) 

0. 539 

(0 . 009) 

C5 

10 

C5 

10 

pnn  ^An9 

X  .  X/  O 

/A  A7Q> 

n  Q4A 

V  U  ,  Ui70  / 

U  •  ^XD 

C5 

C5 

11 

X .  ^ozt 

/A  t\'i'X\ 

((\  n'\(\\ 

V  U  •  llOv  i 

0  495 

(0. 008) 

C5 

12 

C5 

12 

rUUSoU  J. 

U  ,^00 

in  f\oQ\ 

•  A    1  QQ 

tA   A99 \ 
\\J  »  UZZ  J 

A  49A 
U ,  ■I^D 

/A  ni 1 > 

lU • Ull J 

C5 

14 

C5 

14 

irUUOlUl 

t(\  (\k(\\ 

\v . U^U J 

U  .  X 

n  ^sn 

\ u , UUO  i 

C5 

16 

C5 

16 

1  X01 

x*^ox 

/A  A'%Q\ 

A    91  fi 
U  .  ZlO 

tt\  A97^ 
. UZ/ } 

tfi  nnQ^ 

C5 

17 

C5 

17 

pnnc  m  i 

1  CtKO 

(U . UZ9J 

A  ^9Q 

tt\   A9A  ^ 

n  111 

U  .  JXX 

ff\  nnA^ 

C5 

a8 

C5 

18 

W\f\ftA  t\  1 

rUuo<iUi 

x.UoD 

/A  A07\ 

. uz/ ; 

1  AAC 

tt\  AAA\ 

A  AQA 
U  .  •I«JU 

tn  nn7^ 

^  U  .  UU / J 

C5 

19 

C5 

19 

rUUQDUx 

l.ZUU 

/A  AtCX 

A    1 Q") 

/A  A9C\ 
lU . UZO I 

A  A7A 

U  .  /u 

/n  ni n ^ 

lU . UlU  J 

C5 

20 

C5 

20 

rUUQQUl 

A  OXO 

/A    A1 O \ 
. UlO J 

A  Q9Q 

U .  oZo 

tfi  A9C\ 

(U. UZO ; 

A  99R 
U  .  ZZD 

tfi  AA7\ 
VU . UU / ) 

91 

^x 

C5 

21 

irUUQ/Ul 

O  AOX 

/A    AQ1 \ 

A  CCQ 

/A   A9  7 \ 
W  » UZ /  ) 

n  14R 

U .  JHD 

ff\  nn7^ 

^  U . UU ' / 

C5 

22 

C5 

22 

irUUDc'Jl 

1    CI  Q 

/A  AQ7\ 

A  '*9C 
U  .  >3Z0 

ft  417 

ff\  nnQ^ 

C5 

23 

C5 

23 

itUUdoUZ 

A  107 

/A  A1Q\ 

/A  A9C\ 
(U . UZO  J 

A  A41 
U  .  •♦•11 

/I  nnQ^ 

\  * » UUo  J 

C5 

24 

C5 

24 

1    IX  o 

/A    All \ 

1  QUO 

/A  ACQ\ 
VU. UOo ) 

A  QQC 
U  .  09D 

tfi  nnc^ 

(U. UUO J 

C5 

25 

C5 

25 

pnncoHA 
ItUUqoU'i 

/A  AOC\ 

1  940 

/A  n4n^ 

lU . UMU ) 

n  4 4A 

U  .  M  **  D 

( 0 . OuD ) 

C5 

26 

C5 

26 

rUU/UUl 

2.819 

/A  Aoa\ 
(U. UZ9J 

1  con 

lU. U3U ) 

n  9 AO 
U .  Z49 

/A  nn^^ 

\ U . UU J  J 

97 

>7 

pnnoofki 
irUUoZUl 

tt\  AA7^ 

-n  1  CO 
-u . XOO 

ff\  ni7^ 

n  4QA 
U  .  t OD 

ff\  n  1 

^  U . U  X9 / 

C3 

1 

rUUoZUZ 

1*684 

/ n  Aoi \ 

/A  nx <t ^ 

( U . U%3  J 

A  A 

/ A  n9n  ^ 
( U . UZU ) 

C3 

2 

rUUD>7Ul 

1    X  CO 

/ rt  A19\ 
(U  .  U>7ZJ 

A    A  1  7 

/A  nm ^ 
lU .  U>)U  ; 

ft  9<t^ 
U  .  Z3^ 

t (\  ni  A  ^ 
^  u . u  x^  ^ 

C3 

3 

DAD  OA  n 1 

1  <Q7 

X.  3y  / 

/ n  A1A \ 

9    11  1 
Z.  Oil 

/A  noi  ^ 

^  U . UOi  J 

n  91R 

U  .  Z>?D 

t (\  nnc^ 

^  U . UUD  j 

C3 

5 

pnnaxno 

1  QCI 

n  An7 
u .  ou  / 

tfi  f\'\t\ 

0 . 342 

f n  ni  1  ^ 

^  U  .  U  X  X  J 

C3 

6 

pnnoAA'% 

fUUO'tUi? 

1  QAA 

/A  A")?^ 
lU  .  U<7ZJ 

n  771 

tfi 

n  "^SA 

U .  i7  JO 

/A  ni  1  ^ 

W  ,  U  X  X  J 

C3 

7 

pnn AAAA 

O    1  CO 
Z*  lOO 

1  RAA 
X  .  DUO 

tfi  A4fl^ 
\v  t  U*tO  J 

(0, 008) 

C3 

8 

pnn A&n^ 

1  AAA 
X  .  OoO 

0 . 739 

(0.037) 

0 . 405 

(0,011) 

^  _ 

C3 

9 

pnnA&nA 

1  77 

/A  A7S^ 

1  372 

(0 .040) 

0 . 346 

(C . 009) 



C3 

10 

1  7AQ 

ff\  t\9R\ 

1  1 

(0 .038) 

0 . 286 

(0 . 010) 

C3 

11 

pnnftsol 

0 .901 

(0. 020) 

1,748 

0.049) 

0 . 241 

(0.008) 







C3 

12 

& UUOOU  X 

3.427 

(0.031) 

1. 999 

(0.060 ) 

0 . 100 

(0.005) 







_  _ 

C3 

14 

pnn A701 

1.343 

(0. 020) 

1. 312 

(0.033 ) 

0.205 

(0.009) 







C3 

pnnAAni 

1. 149 

/A  AlQ^ 

1. 322 

(0 . 032 ) 

0. 171 

(0.009) 





C3 

13 

f UUOOU  X 

n  747 

(0  020^ 

1.547 

(0 . 048) 

0.237 

(  0 . 0 10 ) 







C3 

15 

pnnonni 

/A  A7S^ 

1.507 

(0.041) 

0.255 

(0 . 008) 





C3 

16 

pnnoi ni 

1. 794 

/n  n99^ 

1  214 

(0 . 032) 

0. 177 

(0,009) 





C3 

17 

pnnQ9ni 

£  •  xu^ 

/n  n9S^ 

1  712 

(0 .045) 

0 . 239 

(0,007) 





C3 

18 

pnno'^ni 

ITUUCIOV  X 

1  47n 

X  ■  ^  /  u 

1. 693 

(0.041) 

0. 162 

(0.008) 





C3 

19 

)>^nfl&ni 

C  WUO**UX 

I'l  0^7 

(0 . 020) 

0 ,807 

(0  028) 

0.212 

(0.011) 







C3 

20 

pnnosm 

1  792 

(0 . 023) 

1, 719 

(0.043) 

0,210 

(0.007) 

__ 







C3 

21 

pnnsRm 

1.059 

(0 . 049) 

-0  ^  g34 

(0.046) 

0,247 

(0.025) 









C3 

22 

PflnOAOl 

1 .356 

(0 . 028) 

0 .401 

(0,029) 

0.231 

(0.013) 









C3 

24 

pnnflflQi 

1.525 

(0.023) 

1 .298 

(0,037) 

0.  342 

(0,009) 









C3 

25 

pni nnoi 

1.615 

(0. 022) 

1.325 

(0.035) 

0,248 

(0,009) 

_  _ 







C3 

26 

pni  ni  01 

f  U  XU  XwX 

1.231 

(0, 022) 

1, 342 

(0.038) 

0, 310 

(0.009) 





_  _ 



C3 

23 

Pni09Q1 

f  U  X  W  OW  X 

2. 147 

(0.032) 

-2,091 

(0.063) 

0,219 

(0 .026) 

C2 

1 





pn 1 1 nni 
r\j  X  xuu  X 

1 

X  .  X 

/A  A9S^ 

-1 

(0 .040) 

0, 283 

(0.017) 

C2 

2 

__ 







pn  1 1 1  ni 
r\j  X  X  xu  X 

n  AA7 

/A  A9A\ 

-2,265 

(0 . 061 ) 

0,233 

(0.023) 

C2 

3 







„  _ 

pn 1 1 oni 

fU  X  XZUx 

1  AQA 

-1  4^^ 

tfi  f\'ia\ 

0 . 240 

(0 . 015) 

C2 

4 









F011301 

1.226 

(0.023) 

-1.753 

(0.043) 

0.227 

(0.019) 

C2 

5 







F011401 

1.761 

(0.044) 

-0.822 

(0.038) 

0.407 

(0.011) 

C2 

6 

F011A02 

1.006 

(0.043) 

-0.744 

(0.045) 

0.488 

(0.011) 

C2 

7 

F011A03 

1.042 

(0.032) 

-1.157 

(0.046) 

0.435 

(0.014) 

C2 

8 

F011404 

1.243 

(0.036) 

-1.022 

(0.043) 

0.409 

(0.0]4) 

C2 

9 

F011405 

1.216 

(0.048) 

-0.163 

(0.023) 

0.426 

(0.007) 

C3 

3 

C3 

3 

F011A06 

1.155 

(0.027) 

-1.218 

(0.036) 

0.386 

(0.012) 

C3 

4 

C3 

4 

F011407 

1.098 

(0.030) 

-1.021 

(0.036) 

0.453 

(0.011) 

C3 

5 

C3 

5 

F011408 

1.110 

(0.032) 

-0.894 

(0.035) 

0.484 

(0.010) 

C3 

6 

3 

6 

F011501 

1.577 

(0.034) 

-0.747 

(0.030) 

0.237 

(0.010) 

C2 

10 

F011601 

1.388 

(0.035) 

-0.631 

(0.030) 

0.268 

(0.010) 

C2 

11 

F011701 

1.337 

(0.046) 

-0.237 

(0.027) 

0.181 

(0.008) 

C2 

12 

F011801 

1.960 

(0.056) 

-0.379 

(0.031) 

0.202 

(0.008) 

C2 

13 

F011901 

1.353 

(0.025) 

-1.121 

(0.031) 

0.187 

(0.013) 

C2 

14 

F012001 

0.878 

(0.043) 

-0.075 

(0.028) 

0.289 

(0.008) 

C2 

15 

F012101 

1.374 

(0.022) 

-1.53^ 

v0.034) 

0.246 

(0.015) 

C3 

1 

C3 

1 

F012201 

1.352 

(0.020) 

-1.7j4 

(0.036) 

0.186 

(0.01.) 

C3 

2 

C3 

2 

F012501 

1.573 

(0.027) 

-1.013 

(0.027) 

0.273 

(0.011) 

C3 

7 

C3 

7 

655 


Table  F-3  (continued) 


1988  IRT  Parameters,  Civics  Cross-sectional 


KAEP  ID 

A 

S.E. 

H 

S.E. 

C 

P012S02 

1.1A9 

(0.027) 

-0.67A 

(0.02A) 

0.260 

P012601 

1.559 

(0.0A5) 

-0.220 

(0.021) 

0.262 

P012701 

1.506 

(0.03A) 

-0.S8A 

<0.023) 

0.268 

P012801 

1.035 

(0.023) 

-0.5A9 

(0.019) 

0.1A9 

PO 12901 

1.262 

(0.028) 

-0.500 

CO. 020) 

0.172 

P013001 

0.591 

(0.020) 

-0.590 

/,0.025) 

0.238 

P013101 

:.1A8 

(0.021) 

-1.012 

(0.025) 

0.151 

P013201 

1.2AA 

(O.OAl) 

0.961 

(0.0A2) 

0.176 

PO 13301 

0.936 

(0.035) 

O.AAO 

(0.025) 

0.237 

P013A01 

1.359 

(0.052) 

0.037 

(0.023) 

0.216 

P013S01 

1.102 

(0.0A7) 

-0.317 

(0.026) 

0.203 

P013601 

0.861 

(0.039) 

0.263 

(0.025) 

0.238 

PO  13701 

1.289 

(0.0A6) 

0.533 

(0.032) 

0.2A8 

P013801 

2,061 

(0.056) 

0.882 

(0.050) 

0.325 

P013901 

1.035 

(0.0A2) 

0.169 

(0.023) 

0.238 

PO 13902 

1.236 

(0.0A3) 

1.119 

(0.052) 

0.230 

POlAOOl 

2.0A9 

(0.097) 

0.016 

(0.029) 

0.289 

POIAIOI 

1.152 

(0.03A) 

-0.A75 

(0.023) 

0.328 

P01A201 

1.185 

(0.027) 

-0.23 A 

(0.016) 

0.212 

P01A3C1 

0.862 

ro.oi8) 

0.303 

(0.015) 

0.2A0 

POlAAOl 

1.503 

(0.022) 

0.372 

(0.016) 

0.199 

P01A301 

l.llA 

(0.023) 

0.02A 

(0,015) 

0.2A2 

P01A601 

1.681 

(0.028) 

0.2A5 

(0.016) 

0.178 

P01A701 

0.829 

(0.016) 

0.393 

(0.015) 

0.190 

P01A801 

1.A26 

(0.025) 

0.3A9 

(0.019) 

0.3A9 

P01A901 

0.999 

(0.020) 

0.17A 

(O.OIA) 

0.240 

POlSOOl 

0.602 

(0.015) 

0.069 

(0.012) 

0.200 

P015101 

1.511 

(0.021) 

1.752 

(O.OAO) 

0.307 

P01S201 

1.365 

(0.019) 

0.728 

(0.019) 

0.272 

P01S202 

1.562 

(0.039) 

-0.238 

(0.018) 

0.205 

P01S301 

1.057 

(O.OIA) 

0.955 

(0.019) 

0.17A 

P0X5A01 

1.32A 

(0.015) 

1.2A0 

(0.022) 

0.159 

?0155C1 

1.232 

(0.016) 

1.238 

(0.02A) 

0.235 

POlStOl 

2.67A 

(0.021) 

1.037 

(0.02A) 

0.227 

P015701 

0.563 

(O.OIA) 

1.053 

(0.029) 

0.226 

P015702 

0.538 

(O.OIA) 

1.500 

(O.OAA) 

0.21A 

P015703 

1.592 

(0.018) 

1.A52 

(0.029) 

0.262 

P01S704 

J.2A1 

(0.020) 

2.0AA 

(0.0A5) 

0.195 

P01S801 

1.16A 

(0.015) 

0.959 

(0.019) 

0.163 

P01S901 

1.6A1 

(0.021) 

0.617 

(0.018) 

0.223 

P01S902 

2.711 

<0.02A) 

0.779 

(0.022) 

0.181 

P016001 

l.r68 

(0.035) 

0.021 

(0.018) 

0.302 

P016101 

1.3AP 

(0.032) 

-0.163 

<0.C17) 

0.216 

P016201 

1.682 

(0.020) 

0.8A6 

(0.022) 

0.226 

P016301 

1.AA5 

(0.027) 

0.^72 

(0.021) 

0.356 

P016A01 

0.978 

(0.021) 

0.2nA 

(0.015) 

0.232 

F016S01 

1.881 

(0.031) 

0.350 

(0.021) 

0.272 

P016501 

C.779 

(0.020) 

0.052 

(0.015) 

0.279 

P:  6701 

1.992 

(0.02A) 

0.785 

(0.023) 

0.250 

PU16801 

0.885 

(0.&18) 

0.535 

(0.019) 

G.2AA 

P015901 

3.729 

(0.02A) 

1.151 

(0.031) 

O.AOO 

P017001 

C.632 

(0.015) 

1.013 

(0.027) 

0.17A 

P017101 

1.998 

(0.022) 

0.831 

(0.022) 

0.206 

P017201 

1.170 

(0.017) 

0.971 

(0.022) 

0.210 

m7301 

1.809 

(0.022) 

0.85A 

(0.023) 

0.272 

P017A01 

1.188 

(0.022) 

0.233 

(0.015) 

0.172 

P017501 

1.616 

(0.020) 

1.187 

(0.027) 

0.24A 

P0I7601 

1.238 

(0.018) 

1.06A 

(0.025) 

0.281 

P017701 

1.6A9 

(0.020) 

1.580 

(0.035) 

0.2A7 

P017801 

X.690 

(0.021) 

l.AAl 

(0.035) 

0.321 

P017901 

0.701 

(0.016) 

0.8A2 

^0.025) 

0.248 

POlSOOl 

1.508 

(0.017) 

1.222 

(0.02^) 

0.183 

P018101 

1.AA3 

(0.019) 

1.418 

(0.031) 

0.266 

P018201 

0.A06 

(0.021) 

-3.026 

(0.158) 

0.000 

ERIC 


GRADE  A/AGE  9 

GRADE 

8/AGE  13 

GRADE 

12/AGE 

S.E. 

BLOCK  ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

(0.009) 

C3  8 

03 

8 

— 

— 

(0,007) 

C3  9 

03 

9 

— 

— 

(0.008) 

C3  10 

03 

10 

— 

(0.008) 

C3  11 

03 

11 

— 

(0.U08) 

C3  12 

03 

12 

— 

— 

(0.009) 

C3  13 

03 

13 

— 

— 

(0.011) 

C3  lA 

03 

lA 

— 

(0.004) 

C3  15 

Oo 

15 

— 

— 

(0.006) 

C3  16 

03 

16 

— 

— 

(0.011) 

— 

03 

17 

— 

— 

(0.013) 

— 

03 

19 

— 

— 

(0.010) 

— 

C3 

18 

— 

(0.008) 

— 

03 

20 

— 

— 

(0.007) 

— 

03 

21 



(0.010) 

— 

03 

22 

— 

(0.007) 

— 

03 

23 



(0.011) 

— 

03 

24 

— 

(0.011) 

— 

06 

1 

06 

1 

(0.010) 

— 

06 

2 

06 

2 

(0.007) 

— 

06 

3 

06 

3 

(0.006) 

— 

06 

4 

06 

4 

(0.008) 

— 

06 

5 

06 

5 

(0.007) 

— 

06 

6 

06 

6 

(0.007) 

— 

06 

7 

06 

7 

(0.007) 

— 

06 

8 

06 

8 

(0.008) 

— 

06 

9 

06 

0 

(0.008) 

— 

06 

10 

06 

10 

(O.OOA) 

— 

06 

11 

06 

11 

(0.006) 

— 

06 

12 

06 

12 

(0.010) 

— 

06 

13 

06 

13 

(0.005) 

— 

06 

lA 

06 

14 

(O.OOA) 

— 

06 

22 

06 

22 

(0.005) 

-- 

06 

16 

06 

16 

(0.004) 

— 

06 

17 

06 

17 

(0.006) 

-- 

06 

18 

06 

18 

(0.005) 

— 

06 

19 

06 

19 

(O.OOA) 

-- 

06 

20 

06 

20 

(O.OOA) 

:6 

21 

06 

21 

(0.005; 

— 

C6 

*5 

06 

15 

(0.006) 

OS 

23 

06 

23 

(0,0('5^ 

— 

C6 

24 

06 

2A 

(c.coe) 

— 

07 

1 

07 

1 

(0.009) 

— 

07 

2 

07 

2 

(0.005) 

— 

07 

3 

07 

3 

(0.007) 

— 

07 

A 

07 

4 

(0.008) 

— 

07 

5 

07 

5 

(0.007) 

-- 

07 

6 

07 

6 

(0.008) 

— 

07 

07 

7 

(0.005) 

07 

07 

8 

(0.007) 

07 

9 

07 

9 

(0.005) 

07 

10 

07 

10 

(0.006) 

07 

11 

07 

11 

(0.005) 

07 

12 

07 

12 

(0.005) 

07 

13 

07 

13 

(0.005) 

07 

14 

07 

14 

(0.007) 

07 

15 

07 

15 

(0.0G5) 

07 

16 

07 

16 

(0.005) 

C7 

1* 

07 

17 

(0.004) 

07 

18 

07 

18 

(0.005) 

07 

19 

07 

19 

(0.006) 

07 

20 

07 

20 

(O.OOA) 

07 

21 

07 

21 

(0.005) 

07 

22 

07 

22 

(0.000) 

08 

1 

08 

1 

656 


Table  F-3  (continued) 


1988  IRT  Parameters,  Civics  Gross -se(*.tional 


GRADE  4 /AGE  9 

GRADE  8/AGE  13 

GRADE 

12/AGE 

HASP  ID 

S.E. 

B 

S.E. 

C 

S.E. 

BLOCK  ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

F018202 

0.861 

(0.014) 

0.685 

(0.017) 

0.000 

(0.000) 

C6 

1 

C8 

1 

F018301 

0.996 

(0.035) 

-0.724 

(0.037) 

0.218 

(0.021) 

— 

— 

C4 

2 

F016401 

1.616 

(0.026) 

0.928 

(0.032) 

0.419 

(0.008) 

—         ~ ' 

— 

C4 

3 

P016S01 

1.309 

(0.047) 

-0.395 

(0.032) 

0.249 

(0.016) 

— 

— 

C4 

4 

F01S601 

0.950 

(0.020) 

0.450 

(0.022) 

0.228 

(0.011) 

— - 

-- 

— 

C4 

5 

F0i3701 

1.502 

(0.023) 

0.751 

(0.02/) 

0.305 

(0.009) 

■*  ~  — 

-- 

-- 

C4 

6 

F018801 

1.067 

(0.022) 

0.955 

(0.032) 

0.419 

(0.006) 

~~         — — 

-- 

C4 

7 

F016901 

0.511 

(0.016) 

0.353 

(0.020) 

0.216 

(0.011) 

**- 

C4 

8 

F019001 

1.322 

(0.018) 

0.748 

(0.022) 

0. 153 

(0.009) 

— —         — — 

-- 

C4 

9 

POIGIOI 

1.935 

(0.023) 

0.864 

(0.025) 

0.180 

(0.008) 

— 

C4 

10 

FOiezoi 

1.061 

(0.018) 

1.336 

(0.032) 

0.257 

(0.007) 

C4 

11 

F019301 

0.799 

(0.019) 

0.245 

(0.019) 

0. 197 

(0.012) 

1  *> 

F019A0X 

1.853 

(0.022) 

1.013 

(0.028) 

0.284 

(0.008) 

C4 

13 

F019SC1 

1.A63 

(0.021) 

0.921 

(0.027) 

0.27^ 

(0.008) 

C4 

14 

F019601 

2.1A8 

(0.021) 

1.100 

(0.028) 

n.2lo 

(0.007) 

C4 

15 

F019701 

2.713 

(0.027) 

1.096 

(0.033) 

0.297 

(0.007) 

C4 

16 

F019801 

0.772 

(0.020) 

2.422 

(0.070) 

9.157 

(0.005) 

C4 

17 

F019901 

1.238 

(0.019) 

0.687 

(0.025) 

0  209 

(0.008) 

C4 

18 

F020001 

1.470 

(0.018) 

1.271 

(0.026) 

0.188 

(0.007) 

C4 

19 

F020101 

1.61A 

(0.024) 

1.827 

(0.047) 

0.308 

(0.006) 

C4 

20 

F020201 

0.574 

(0.017) 

2.146 

(0.069) 

0.190 

(0.007) 

C4 

21 

F020301 

1.958 

(0.020) 

1.115 

(C.027) 

0.164 

(0,007) 

C4 

22 

F020401 

2.27.8 

(0.025) 

1.636 

(0.047) 

C  229 

(0.005) 

C4 

23 

F020S01 

1.922 

(0.021) 

1.613 

(0.035) 

0.195 

(0.006) 

C4 

24 

F020601 

0.677 

(0.029) 

2.947 

(0.135) 

0.251 

(0.006) 

C4 

25 

F020701 

1.691 

(0.027) 

0.704 

(0.026) 

0.324 

(0.007) 

C5 

9 

C5 

9 

P021301 

0.840 

(0.031) 

-0.676 

(0.035) 

0.239 

(0.019) 

C4 

1 

ERIC 
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Table  F-4 


1988  IRT  Parameters,  Civics  Trend,  Age  13 


NAEP  ID 

A 

S.E. 

B 

FOOOlOl 

0.531 

(0.057) 

0.212 

F000201 

0.421 

(0.048) 

-1.449 

FOOO^Cl 

0.168 

(0.037) 

-0.185 

F000501 

0.715 

(0.053) 

-2.550 

F000601 

0.393 

(0.057) 

-2.956 

F000801 

0.885 

(0.054) 

-1.638 

F000802 

0.826 

(0.070) 

-0.506 

FOOOQOl 

0.413 

(0.050) 

-1.529 

FOOlOOl 

0.621 

(0.071) 

-0.805 

FOOllOl 

0.394 

(0.054) 

-3.152 

F001201 

0.232 

(0,037) 

-2.410 

F001301 

0.700 

(0.061) 

0.331 

FOOl^Ol 

0.530 

(0.051) 

-1.558 

F001501 

0.718 

(0.086) 

-0.195 

F001601 

0.415 

(0.052) 

-2.313 

F001701 

0.6C7 

(0.079) 

-0.769 

POO  mi 

0.415 

(0.048) 

-1.627 

F001802 

0.931 

(0.105) 

-0.343 

Fooigol 

0.519 

(0.057) 

2.265 

F002101 

1.066 

(0.077) 

1.622 

F002201 

0.3b3 

(0.050) 

-2.438 

F002301 

0.831 

(0.056) 

-0.527 

F002^01 

0.516 

(0.056) 

3.028 

F002501 

0.764 

(0.050) 

2.057 

F002o01 

0.515 

(0.050) 

-0.517 

F002701 

0.530 

(0.048) 

0.037 

F0027C2 

C.309 

(0.045) 

2.316 

F002901 

0.536 

(0.063) 

-0.814 

F003001 

0.496 

(0.051) 

0.26C 

F003201 

0.598 

(0.052) 

0.727 

F003^01 

0.880 

(0.063) 

2.057 

F003501 

0.523 

(0.066) 

3.496 

P003601 

0.819 

(0.054) 

1.907 

F003701 

0.628 

(0.051) 

-0.004 

F003801 

0.629 

(0.068) 

2.658 

FOO^OOl 

0.767 

(0.082) 

-0.819 

F00^002 

0.349 

(0.040) 

-0.707 

F004003 

1.917 

(0.086) 

-1.322 

F004004 

0.329 

(0.064) 

2.782 

F004005 

0.573 

(0.064) 

1.007 

F004007 

0.822 

(0.087 

1.966 

F004008 

0.927 

(0.073> 

-1.153 

F004010 

1.894 

(0.084) 

-1.375 

FOO^Oll 

0.464 

(0.055) 

0.764 

F00^012 

0.568 

(0.085) 

2.629 

F004101 

0.300 

(0.060) 

-0.958 

F004201 

0.904 

(0.066) 

-0 

FOO^^Ol 

0.292 

(0.048) 

-1.817 

FOO^SOl 

0.860 

(0.085) 

1.698 

F004601 

0.370 

(0.055) 

-1.003 

F00^701 

0.372 

(0.U50) 

-1.013 

F004901 

0.605 

(0.054) 

-0.554 

FOOSOOl 

0.921 

(0.079) 

2.293 

F00S301 

0.587 

(0.055) 

0.321 

FOOSOOl 

0.789 

(0.064) 

-1.649 

F005S01 

0.508 

(0.049) 

-1.922 

F00SS02 

0.859 

(0.078) 

-2.671 

FOOSSO^ 

1.082 

(0.081) 

-2.432 

F00SS05 

0.505 

(0.055) 

-1.920 

F005601 

0.844 

(0.057) 

-1.505 

F005801 

0.533 

(0.051) 

-1.771 

F005802 

0.057 

(0.084) 

1.932 

F005803 

0.649 

(0.073) 

0.378 

F006001 

0.487 

(0.088) 

3.509 

AGE 

13 

S.E. 

C 

S.E. 

BLOCK 

ITEM 

(0.044) 

0.455 

(0. 

014) 

C9 

1 

(0.171) 

0.463 

(0. 

021) 

C9 

2 

(0.049) 

0.472 

(0. 

015) 

C9 

4 

(0.194) 

0.296 

(0. 

040) 

C9 

5 

(0.43j) 

0.454 

(0.035) 

C9 

6 

(0.112) 

0.257 

(0.034) 

C9 

(0.057) 

0.236 

(0. 

021) 

C9 

9 

(0.191) 

0.456 

(0. 

024) 

C9 

10 

(0.104) 

0.492 

(0. 

022) 

C9 

11 

(0.435) 

0.446 

(0. 

036) 

C9 

12 

(0.382) 

0.466 

(0. 

024) 

CP 

13 

(0.049) 

0.406 

(0. 

013) 

C9 

14 

(0.153) 

0.440 

(0.023) 

C9 

15 

(0  054) 

0.429 

(0.019) 

C9 

10 

(0.292) 

0.442 

(0.031) 

C9 

17 

(0.105) 

0.475 

(0. 

022) 

C9 

18 

(0.192) 

0.418 

(C. 

026) 

C9 

19 

(0.066) 

0.418 

(0. 

021) 

C9 

20 

(0.258) 

0.290 

(0.011) 

C9 

Zi 

(0.154) 

0.418 

(0. 

012) 

C9 

23 

(0.321) 

0.436 

(0. 

031) 

C9 

24 

(0.047) 

0.236 

(0. 

018) 

C9 

25 

(0.335) 

0.148 

(0. 

008) 

C9 

26 

(0.147) 

0.198 

(0. 

008) 

C9 

27 

(0.060) 

0.239 

(0.021) 

C9 

28 

(0.029) 

0.291 

(0.015) 

C9 

29 

(0.336) 

0.296 

(0 

OiO) 

C9 

30 

(0.106) 

0.447 

(0 

023) 

C9 

32 

(0.044) 

0.250 

(0. 

017) 

C9 

33 

(0.072) 

0.226 

(0 

013) 

CI 

1 

(0.171) 

0.234 

(0 

010) 

CI 

2 

(0.450) 

0.172 

(0 

008) 

CI 

3 

(0.143) 

0.173 

(0 

010) 

CI 

4 

(0.033) 

0.206 

(0 

017) 

CI 

5 

(0.300) 

0.221 

(0 

010) 

CI 

6 

(0.100) 

0.438 

(0 

025) 

CI 

7 

(0.089) 

0.403 

(0 

020) 

CI 

8 

(0.106) 

0.333 

(0 

035) 

CI 

9 

(0.548) 

0.450 

(0 

013) 

CI 

10 

(0.126) 

0.436 

(0 

014) 

CI 

11 

(0.238) 

0.463 

(0 

012) 

CI 

12 

(0.10^) 

0.394 

(0 

028) 

CI 

13 

(0.105) 

0.343 

(0 

035) 

CI 

14 

(0.102) 

0.445 

(0 

015) 

CI 

15 

(0.409) 

0.434 

(0 

013) 

CI 

16 

(0.081) 

0.218 

(0 

026) 

CI 

18 

(0.066) 

0.207 

(0 

021) 

CI 

19 

(0.304) 

0.460 

(0 

026) 

CI 

20 

(0. 198) 

0 . 499 

(0 

013) 

CI 

91 

iSi 

(0.156) 

0.468 

(0 

023) 

CI 

22 

(0.142) 

0.440 

(0 

023) 

CI 

23 

(0  064) 

0.369 

(0 

.022) 

CI 

25 

(0.232) 

0.293 

(0.010) 

CI 

27 

(0.048) 

0.417 

(0 

.013) 

CI 

1 

(0.142) 

0JA7 

(0 

.029) 

CI 

2 

(0.192) 

0.458 

(0 

.028) 

CI 

5 

(0.258) 

0.427 

(0 

049) 

CI 

6 

(0.214) 

0.457 

(0 

.051) 

CI 

7 

(0.215) 

0.436 

(0 

.032) 

CI 

8 

(0.n2) 

0.275 

(0 

.032) 

CI 

3 

(0.175) 

0.431 

(0 

.025) 

CI 

10 

(0.216) 

0.462 

(0 

.010) 

CI 

11 

(0.061) 

0.462 

(0 

.014) 

CI 

12 

(0.647) 

0.385 

(0 

.011) 

CI 

15 
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Table  F-4  (continued) 


1988  IRT  Parameters,  Civics  Trend,  Age  13 

AGE  X3 


MAEP  ID 

S.E. 

B 

S.E, 

C 

S.E. 

BLOCK 

ITEM 

P006XOX 

0.93  A 

(0.X04) 

0.409 

(0.079) 

0.534 

(0.0X5) 

cx 

X6 

P00620X 

0.980 

(0.XX8) 

-0.034 

(0.058) 

0.526 

(0.0X7) 

cx 

X7 

P00630X 

0.763 

(0.063) 

0.933 

(0.093) 

0.3X7 

(0.0X3) 

CX 

X8 

P00640X 

0.590 

(0.06X) 

X.005 

(0.XX4) 

0.427 

(0.0X3) 

cx 

X9 

P006S0X 

0.752 

(0.096) 

0.273 

(0.059) 

0.490 

(0.0X4) 

cx 

20 

P00660X 

0.4X7 

(0.042) 

0.935 

(0.099) 

0.247 

(0.0X3) 

cx 

2X 

P00670X 

0.696 

(0.07X) 

0.X89 

(0.044) 

0.259 

(0.0X7) 

cx 

22 

P00680X 

0.748 

(0.079) 

0.036 

(0.049) 

0.390 

(0.0X9) 

cx 

23 

P006802 

0.237 

{0.039) 

-0.567 

(O.XOX) 

0.466 

(0.019) 

cx 

24 

F006803 

0.565 

(0.070) 

3.069 

(0.394) 

0.288 

(0.009) 

CI 

25 

P006804 

0.744 

(0.062) 

X.522 

(0.X42) 

0.423 

(0.0X0) 

cx 

26 

P00720X 

0.79X 

(0.058) 

0.X8X 

(0.039) 

0.203 

(0.0X6) 

cx 

29 

P00730X 

0.434 

(0.060) 

-0.403 

(0.068) 

0.485 

(0.0X7) 

cx 

30 

P00740X 

0.429 

(0.049) 

-X.774 

(0.205) 

0.442 

(0.023) 

cx 

3X 

P00750X 

0.5X4 

(0.056) 

-2.363 

(0.264) 

0.44S 

(0.035) 

cx 

32 

P007502 

0.776 

(0.095) 

0.074 

(0.052) 

0.487 

(0.0X6) 

cx 

33 

P007503 

0.559 

(0.076) 

-0.242 

(0.058) 

0.492 

(O.0X9) 

cx 

34 

P02070X 

0.530 

(0.056) 

0.223 

(0.043) 

0.288 

(0.0X7) 

cx 

9 

P02XX0X 

0.509 

(0.037) 

-X.882 

(0.X42) 

0.000 

(0.000) 

cx 

28 
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Table  F-5 


1988  IRT  Parameters,  Civics  Trend,  Age  17 


AGE  17 


HAEP  ID 

A 

u 

S.E. 

B 

S.E. 

S  E 

BL 

ITEM 

FOOOlOl 

0.683 

(0.066) 

-O.AOS 

(O.OSS/ 

0. 502 

(0.01^) 

Cit 

97 

FC00201 

0.S26 

(0.0S3) 

"1. 490 

(0. 1S4 ) 

0 . 479 

(0 . 021 ) 

CO 

Zo 

POOOAOl 

0. 17* 

(0.033) 

"0. 160 

(0.04S) 

0. 525 

(0.014 ) 

Ov 

P000801 

1, 335 

(O.OSO) 

~1. 886 

(0. 098) 

(0. 040) 

ro 

P000802 

1.  Ill 

(O.OSO) 

-I.SIO 

(0. 08S) 

(0 . 032) 

CQ 

00 

P003201 

0. 622 

(0.0A7) 

"0.414 

(0. 042) 

\\)  *\)  Lt  } 

1 

P003A01 

I.OIA 

(0.087) 

0.239 

(0. 049) 

0 . 266 

(0. 015) 

n 

i. 

P003501 

1. 262 

(0.0S9) 

.013 

(0.068) 

\  U  .  \JX\J  J 

0 

P003601 

1.008 

(0.068) 

0.421 

(O.OSl) 

0. 194 

(0.014 ) 

n 

t, 

s 

P003701 

0. 559 

(O.OSO) 

-X. 089 

(0. 090) 

n  7An 
u .  ^mj 

\  U  .  \J^J  } 

c 

1. 17A 

(0.079) 

0 . 739 

(0. 073) 

u .  ^ 

c 
D 

P004001 

1. 2AS 

(0.072) 

"1 . S78 

(0. 118) 

\\J  *  \JOO  J 

/ 

PO0A002 

0.  A21 

(0.0A7) 

-1. 761 

(0.200) 

\v  * U£D / 

U  k 

0 

P004003 

1. 327 

(0.063) 

*"1. 9S7 

(0 . 128) 

tn  t\k'y\ 

\v  * v^C) 

V  i 

n 

D 

POOAOOA 

0.  A7S 

(0.0S6) 

"0. 183 

(0. 050) 

iU 

P00A005 

0. 933 

(0.079) 

0.2S3 

(0 . 060) 

u .  Him 

\V  * VXJ  J 

11 

P00A007 

1. 188 

(O.IOS) 

0. 403 

(0. 074 ) 

0 . 463 

(0 . 014 ) 

Xl, 

P004008 

0.988 

(0.062) 

"1 . 768 

(0. 129) 

U  .  ■* ^9 

(0. 035) 

10 

P004010 

1.39A 

(0.067) 

-1.883 

(0. 124) 

0 . 423 

(0. 041) 

CI 

\  X 

1^ 

P004011 

0.571 

(0.0S7) 

0.403 

(0. 062) 

(0. 015) 

13 

P00A012 

O.SOA 

(0.0S7) 

0.674 

(0. 091) 

u .  ■?/  o 

(0. 015) 

CI 

ID 

P00A201 

1. 106 

(O.OSO) 

-1.S48 

(0. 084 ) 

0 . 277 

(0. 030) 

Id 

POOAAOl 

O.SOS 

(0.069) 

-1.418 

(0. 199) 

0. 502 

(0. 027) 

CI 

9n 

P004S01 

l.OAS 

(0. 122) 

-0.0S6 

(0. 060) 

0 . 494 

(0. 018) 

U  X 

91 

€mX 

P00A601 

0.778 

(0.08A) 

-1. 104 

(0. 132) 

0.511 

(0. 026) 

CI 

99 

P00A701 

0,S77 

(0.063) 

-1. 164 

(0. 135) 

0. 462 

(0. 025) 

CI 

9*^ 
£0 

POOAQOl 

0.S97 

(0.062) 

-1,539 

(0. 166) 

0. 452 

(0. 028) 

CI 

9*? 

P005001 

1. 26A 

(0.07A) 

0.776 

(0.073) 

0. 277 

(0. 012) 

CI 

•^9 
0£ 

P005101 

0.90A 

(0.083) 

-0.542 

(0.069) 

0.405 

(0. 022) 

n 

^  X 

27 

POOS  102 

1.3A2 

(0.091) 

-0.509 

(0. 061) 

0.345 

(0. 019) 

CI 

^0 

POOS  103 

1.276 

(0.101) 

0.279 

(0,058) 

0.371 

(0. 014 ) 

CI 

29 

POOSlOA 

1.788 

(0.123) 

1.442 

(0. 183) 

0.559 

(0. 010) 

CI 

30 

POOSIOS 

1.  A21 

(0.120) 

-0.233 

(0.058) 

0.415 

(0.017) 

CI 

31 

PO0S301 

1.269 

(0.089) 

-0.923 

(0.085) 

0.505 

(0.022) 

CI 

POOSAOl 

1.086 

(0.06S) 

-1.986 

(0. 137) 

0.479 

(0.038) 

CI 

POOSSOl 

1.A07 

(0.072) 

-1.663 

(0. 116) 

0.437 

(0.039) 

CI 

5 

P005S02 

0.699 

(0.070) 

-3.130 

(0.326) 

0.462 

(0.051) 

CI 

P00SS04 

0.936 

(0.123) 

-3.212 

(0. 458) 

0. 468 

(0 , 062) 

CI 

P00SS05 

0.8S1 

(0.076) 

-2.01S 

(0. 19'i) 

0. 460 

(0.0*2) 

CI 

3 

P00S601 

1.S73 

(0.06A) 

-1.697 

(0. 100) 

0.232 

(0  041) 

CI 

3 

POOSSOl 

0.S77 

(0.0S3) 

- 1 .  S*/3 

(0. 151) 

0.493 

(0.023) 

CI 

10 

P00S802 

0.72A 

(0.07S) 

0.23S 

(0.051) 

0. 466 

(0.014) 

CI 

PO0S803 

0.936 

(0.077) 

-0.701 

(0.0V2) 

0. 461 

(0.020) 

CI 

P00S80A 

0.620 

(0.092) 

3.0S1 

(0. 473) 

0. 408 

(0.009) 

CI 

13 

P006101 

0.8AS 

(0.072) 

-0.839 

(0.087) 

0.443 

(0.025) 

CI 

16 

P006201 

1.037 

(0.083) 

-0.750 

(0.079) 

0.463 

(0.023) 

CI 

17 

P006301 

1.29A 

(0.100) 

-0.429 

(0.059) 

o!314 

(0.G20) 

CI 

18 

P006A01 

O.SOA 

(O.OSS) 

0.3S1 

(0.056) 

0.475 

(0.014) 

CI 

19 

P006S01 

0.899 

O.08A) 

-0.67S 

(0.077) 

0.506 

(0.019) 

CI 

20 

PC06601 

0.SS2 

(0.0A3) 

-0.187 

(0.031) 

0.247 

(0.016) 

CI 

21 

PC06701 

0.968 

(0.072) 

-0.878 

(0.078) 

0.242 

(0.027) 

CI 

22 

P006801 

0.8A3 

^0.080) 

-0.721 

(0.085) 

0.431 

(0.024) 

CI 

23 

P006802 

0.381 

(0.0S3) 

-0.363 

(0.066) 

0.498 

(0.018) 

CI 

24 

P006803 

1.061 

(0.086) 

1.607 

(0.128) 

0.369 

(0.010) 

CI 

25 

P006804 

0.887 

(0.063) 

0.492 

(0.055) 

0.389 

(0.012) 

CI 

26 

P007201 

1.A90 

(0.090) 

-0.704 

(0.065) 

0.244 

(0.024) 

CI 

29 

P007301 

0.839 

(0.086) 

-C.840 

(0.099) 

0.530 

(0.021) 

CI 

30 

P007A01 

0.963 

(0.060) 

-l.SSS 

(0.110) 

0.473 

(0.029) 

CI 

31 

P007301 

0.410 

(0.063) 

-3.6S1 

(0.562) 

0.476 

(0.041) 

CI 

32 

P007502 

O.SOS 

(0.0S3) 

-1.014 

(0.114) 

0.473 

(0.022) 

CI 

33 

P007303 

0.92A 

(0.078) 

-1.071 

(0.105) 

0.479 

(0.027) 

CI 

34 

P00820i 

0.923 

(0.068) 

-1.318 

(0.112) 

0.506 

(0.027) 

eg 

1 

P008202 

1.300 

(0.076) 

-1.343 

(0.106) 

0.476 

(0.030) 

C9 

2 

P008301 

0.8S2 

(0.0S3) 

-1.108 

(0.080) 

0.252 

(0.026) 

C9 

3 

694 
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Table  F-5  (continued) 


1988  IRT  Parameters,  Civics  Trend,  Age  17 


HAEP  ID 

A 

S.E. 

P008401 

1.166 

(0.068) 

F008402 

2.414 

(0.284) 

F008403 

1.405 

(0.105) 

F008404 

2.417 

(0.100) 

P00840S 

1.664 

(0.186) 

P008406 

1.277 

(0.06g) 

F008407 

1.637 

(0.108) 

F008501 

0.7g6 

(0.061) 

P008601 

1.271 

(0.065) 

F008701 

0.888 

(0.058) 

Foogool 

1.482 

(0.066) 

Foogioi 

0.g48 

(0.061) 

Foogzoi 

1.138 

(0.057) 

Foogsol 

0.886 

(0.059) 

FOog^oi 

0.474 

(0.03g) 

Foogsol 

1.027 

(0.055) 

Foogeoi 

0.653 

(O.060) 

F009801 

0.669 

(0.055) 

FOOgQOl 

1.3g7 

(0.069) 

FOIOIOI 

0.550 

(0.045) 

FQ20701 

1.052 

(0.089) 

P021001 

1.071 

(0.049) 

P021101 

0.5g3 

(0.042) 

B  S.E.  C 

2.014  (0.154)  0.216 

0.010  (0.074)  0.393 

-0.131  (0.051)  0.371 

1.215  (0.125)  0.364 

-0.067  (0.064)  0.463 

0.763  (0.071)  0.348 

0.401  (0.071)  0.353 

0.638  (0.066)  0.279 

2.356  (0.164)  0.08g 

0.816  (0.070)  0.245 

0.874  (0.067)  0.211 

0.g94  (0.082)  0.201 

1.310  (0.088)  0.188 

1.579  (0.124)  o.igg 

-1.105  (0.096)  0.228 

1.579  (0.105)  0.168 

-3.201  (0.303)  0.236 

-0.991  (0.090)  0.239 

0.420  (0.048)  0.315 

0.595  (0.058)  0.295 

-0.435  (0.056)  0.267 

-1.238  (0.069)  0.000 

-2.269  (0.165)  0.000 


AGE 

17 

S.E. 

BLOCnC 

ITEM 

(0.009) 

C9 

5 

(0.015) 

C9 

6 

(0.017) 

C9 

7 

(0.011) 

C9 

8 

(0.017) 

C9 

g 

(0.013) 

C9 

10 

(0.015) 

C9 

11 

(0.015) 

C9 

12 

(0.007) 

C9 

14 

(0.012) 

eg 

4 

(0.011) 

eg 

16 

(0.013) 

eg 

17 

(0.010) 

eg 

18 

(0.012) 

C9 

ig 

(0.024) 

eg 

20 

(0.009) 

eg 

21 

(0.047) 

eg 

22 

(0.026) 

eg 

24 

(0.011) 

eg 

25 

(0.012) 

eg 

23 

(O.021) 

ei 

g 

(0.000) 

ei 

33 

(0.000) 

ei 

34 
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Table  F-6 


1988  IRT  Parameters,  U.S.  History  Cross -sectional 


GRADE  4 /AGE  9 

GRfiDE  8/AGE  13 

GRADl 

12/AGE 

KAEP  ID 

S.E. 

B 

S.E. 

C 

S.E. 

BLOCX  ITEM 

BLOCK  ITEM 

BLoa: 

ITEM 

HOOOlOl 

1.562 

(0.014) 

1.061 

(0.019) 

0.223 

(0.005) 



H3 

1 

H000201 

1.575 

(0.014) 

1.549 

(0.024) 

0.205 

(0.004) 

_  ^ 

_  _         _  _ 

H3 

3 

H000301 

0.760 

(0.016) 

0.019 

(0.014) 

0.272 

(0.009) 



_  - 

H3 

HOOOAOl 

1.558 

(0.021) 

1.679 

(0.039) 

0.496 

(0.004) 

  _  _ 

_  _         _  _ 

H3 

5 

B000601 

0.723 

(0.012) 

0.529 

(0.015) 

0.151 

(0.008) 





B3 

6 

E000801 

1.197 

(0.034) 

-0.459 

(0.024) 

0.286 

(0.013) 

  _  _ 

H3 

7 

H000901 

1.687 

(0.013) 

1.444 

(0.021) 

0.171 

(0.004) 

_  _ 

B3 

8 

HOOllOl 

1.561 

(0.016) 

1.036 

(0.020) 

0.297 

(0.005) 

  _  _ 

H3 

9 

H001301 

2.153 

(0.022) 

0.683 

(0.021) 

0.334 

(0.006) 



  _  _ 

B3 

13 

HOOlAOl 

0.624 

(0.012) 

0,458 

(0.015) 

0.230 

(0.008) 

,  _         _  ^ 

  _  _ 

B3 

14 

HOOlSOl 

1.257 

(0.014) 

0.602 

(0.015) 

0. 144 

(0.007) 

m,—   

— —         — » 

B3 

15 

H001701 

1.532 

(0.014) 

1.301 

(0. 021) 

0.218 

(0.005) 



  _  _ 

B3 

17 

H001801 

1.452 

(0.016) 

1.833 

(0.031) 

0.245 

(0.004) 

  „  _ 

  _  _ 

B3 

18 

E001901 

0.774 

(0.012) 

0.990 

(0.020) 

0.196 

(0.006) 

^  _   

  _  _ 

19 

H002001 

1.058 

(0.013) 

0.843 

(0.017) 

0.215 

(0.006) 

  ^  _ 

  _  _ 

B3 

20 

H002101 

1.309 

(0.013) 

1.422 

(0.023) 

0.229 

(0.005) 



  _  _ 

B3 

21 

H002201 

1.064 

(0.016) 

0.494 

(0.017) 

0.299 

(0.008) 



  _  _ 

B? 

22 

H002301 

2.722 

(0.024) 

2.094 

(0.049) 

0. 198 

(0.003) 



— —         _  _ 

B3 

23 

H002601 

1.964 

(0.016) 

1.483 

(0.025) 

0.253 

(0.004) 

__ 

  _  _ 

B3 

24 

H002701 

1.188 

(0.013) 

1.466 

(0.023) 

0.192 

(0.005) 





B3 

25 

H003001 

1.201 

(0.016) 

0.078 

(0.009) 

0.232 

(0.005) 

H5  1 

B5 

1 

H003101 

1.119 

(O.Oil) 

0.872 

(0.012) 

0.152 

(0.003) 



fl5  2 

B5 

2 

H003201 

1.825 

(0.013) 

1.594 

(0.022) 

0.204 

(0.002) 

fl5  3 

B5 

3 

H003301 

0.818 

(0.010) 

0.S33 

(0.015) 

0.202 

(0.004) 

H5  4 

H5 

H003601 

1.555 

(0.012) 

1.113 

(0.016) 

0.261 

(0.003) 

H5  7 

B5 

7 

H003701 

1.467 

(0.016) 

1.778 

(0.031) 

0.349 

(0.003) 

B5  8 

H5 

8 

H003801 

1.538 

(0.012> 

1.624 

(0.021) 

0.164 

(0.002) 



H5  9 

H5 

9 

HOOAOOl 

0.937 

(0.011) 

1.234 

(0.018) 

0.259 

(0.003) 



fl5  10 

H5 

10 

H004301 

0.649 

(0.015) 

2.217 

(0.056) 

0.384 

(0.003) 



B5  12 

G5 

12 

1.359 

(0.014) 

0.761 

(0.014) 

0.327 

(0.004) 



fl5  13 

H5 

13 

H004S01 

0.526 

(0.011) 

-0.775 

(0.018) 

0.204 

(0.007; 

B5  14 

H5 

14 

R004S02 

0.723 

(0.011; 

0.166 

(0.008) 

0.191 

(0.005) 

H5  15 

H5 

15 

£004601 

0.747 

(0.009) 

1.097 

(0  AB) 

0.142 

(0.003) 

B5  17 

H5 

17 

H004901 

0.825 

(0.011) 

0.180 

(0.008) 

0.163 

(0.005) 

H5  18 

B5 

18 

H005001 

2.204 

(0.018) 

0.577 

(0.014) 

0.280 

(0.004) 



H5  19 

H5 

19 

B00S201 

1.451 

(0.011) 

0.970 

(0.013) 

0.191 

(0.003) 



H5  22 

H5 

22 

H005301 

0.539 

(0.010) 

1.340 

(0.026) 

0.211 

(0.004) 

H5  23 

H5 

23 

B00S401 

1.318 

(0.013) 

0.573 

(0.011) 

0.216 

(0.004) 



B5  24 

H5 

24 

H005S01 

1.526 

(0.016) 

1.865 

(0.031) 

0.284 

(0.003) 

H5  25 

H5 

25 

H005801 

0.882 

(0.014) 

•0.130 

(0.009) 

0.231 

(0.006) 



fl5  26 

H5 

26 

B003S01 

1.632 

(0.015) 

0.939 

(0.018) 

0.215 

(0.005) 

R4 

1 

H006001 

1.470 

(0.017) 

0.541 

:0.016) 

0.205 

(0.007) 

— 

B4 

2 

H006101 

1.367 

(0.029) 

-0.138 

(0.018) 

0.228 

(0.0X1) 

-_ 

B4 

3 

H006401 

1.551 

(0.013) 

1.920 

(0.027) 

0.113 

(0.003) 



-_ 

B4 

6 

B00660I 

1.445 

(0.013) 

1.442 

(0.023) 

0.223 

(0.004) 





B4 

7 

B006701 

1.858 

(0.020) 

2.208 

(0.044) 

0.164 

(O.OOJ) 





fl4 

d 

H006801 

1.864 

(0.018) 

0.771 

(0.020) 

0.303 

(0.006) 



-_ 

B4 

9 

H007001 

1.845 

(0.015) 

1.090 

(0.020) 

0.231 

(0.005) 



— 

H4 

13 

H007101 

1.159 

(0.017) 

2.117 

(0.041) 

0.250 

(0.004) 

B4 

10 

H007102 

1.787 

(0.015) 

1.037 

(0.019) 

0.238 

(0.005) 

B4 

11 

B007103 

1.196 

(0.016) 

0.790 

(0.019) 

0.339 

(0.006) 

H4 

12 

H007201 

0.671 

(0.013) 

1.467 

(0.033) 

0.314 

(0.005) 

B4 

14 

H007301 

1.757 

(0.015) 

1.882 

(0.029) 

0.153 

(0.003) 

B4 

15 

H007401 

1.795 

(0.016) 

1.905 

(0.033) 

0.219 

(0.004) 

B4 

18 

H007501 

1.342 

(0.013) 

1.201 

(0.020) 

0.219 

(0.005) 

B4 

17 

H007601 

1.341 

(0.014) 

1.626 

(0.026) 

0.233 

(0.004) 

B4 

18 

H007701 

1.965 

(0.021) 

0.684 

(0.020) 

0.312 

(0.006) 

B4 

19 

H007801 

1.129 

(0.012) 

1.083 

(0.018) 

0.164 

:o.oo5) 

B4 

20 

B008101 

0.171 

(0.011) 

1.072 

(0.C19) 

0.145 

;n.oo6) 

B4 

22 

H008201 

0.878 

(0.013) 

1.846 

(0.032) 

0.208 

(0.004) 

B4 

23 

H008S01 

1.14C 

(0.031) 

-0.624 

(0.027) 

0.235 

(0.014) 

B4 

24 

H008701 

1.362 

(0.014) 

1.588 

(0.025) 

0.233 

(0.004) 

B4 

25 

BC08801 

1.094 

(0.013) 

1.143 

(0.021) 

0.285 

(0.005) 

B3 

2 

Hooasoi 

2.411 

(0.015) 

1.046 

(0.017) 

0.292 

(0.003) 

fl5  5 

BS 

5 

ERIC 
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Table  F-6  (continued) 


1988  IRT  Parameters,  U.S.  Hist'^rv  Cross- sectional 


GRADE 

(/AGE  9 

C>RADE 

8/AGE  13 

GRADE 

1  /AGE 

HAEP  ID 

L 

S  5^ 

B 

S.E. 

c 

S.E. 

Bixxnc 

11^ 

ITEM 

BLOCK 

B009201 

0.813 

(0.014) 

(0.0X6) 

0.254 

(0.008) 

— — 

— — 

— — 

H3 

10 

B009401 

0.684 

(0.014) 

0.295 

(0.014) 

0.238 

(0.008) 

— — 

— — 

— — 

H4 

4 

Boogsol 

1.341 

(C.0J4) 

1.205 

CO. 022) 

0.29g 

(0.005) 

— — 

— — 

B3 

11 

Boogeoi 

2.409 

(0.017) 

1.754 

(0.030) 

0.103 

(0.004) 

-- 

-- 

B3 

12 

Boogeoi 

1.854 

(0.025) 

0.439 

(0.020) 

0.346 

(0.008) 

— — 

— 

**— 

— — 

B3 

16 

B009901 

1.044 

{0.C22) 

-0.324 

(0.018) 

0.214 

(0.012) 

-- 

-- 

B4 

5 

B310101 

1.536 

(0.015) 

0.982 

(0.019) 

0.270 

(0.006) 

— — 

H4 

21 

B010201 

1.125 

(0.013) 

1.091 

(0.018) 

0.362 

(0.003) 

— — 

— — 

H5 

11 

B5 

11 

B010401 

1.787 

(0.012) 

1.403 

(0.017) 

0. 163 

(0.002) 

H5 

16 

H5 

16 

B010801 

0.648 

(0.011) 

1.634 

(0.029) 

0.226 

(0.003) 

— — 

— — 

H5 

20 

B5 

20 

BOllOOl 

1.737 

(0.013) 

0.697 

(0.012) 

0. 145 

(0. 003) 

— — 

H5 

21 

H5 

21 

B011501 

0.761 

(0.010) 

-1.141 

(0.018) 

0.308 

(0.006) 

H2 

1 

B2 

1 

— — 

— " 

B011601 

1.233 

(0.010) 

-1.617 

(0.017) 

0. 166 

(0.008) 

u2 

2 

H2 

2 

B011701 

1.074 

(0.009) 

-2.023 

(0.021) 

0.167 

(0.011) 

H2 

3 

H2 

3 

"— 

Boiieoi 

0.912 

(0.009) 

-1.591 

(0.018) 

0.158 

(0.008) 

H2 

4 

H2 

4 

Bciigoi 

1.412 

(0.020) 

-0.806 

(0.017) 

0.487 

(0.005) 

B2 

5 

H2 

5 

B012001 

0.831 

(0.009) 

-1.619 

(0.020) 

0.202 

(0. 008) 

H2 

6 

H2 

6 

H012101 

2.oog 

(0.024) 

-0.471 

(0.013) 

0.264 

(0.004) 

H2 

7 

H2 

7 

— — 

BO 12201 

1.246 

(0.014) 

-0.641 

(0.012) 

0.213 

(0.004) 

H2 

8 

H2 

8 

B012301 

1.654 

(0.019) 

-0.400 

(0.010) 

0. 170 

(0.003) 

B2 

9 

H2 

9 

B012401 

1.931 

(0.018) 

-0.655 

(0.012) 

0. 184 

(0.004) 

H2 

10 

H2 

10 

—  - 

B012501 

2.601 

(0.032) 

-0.384 

(0.013) 

0.236 

(0.003) 

H2 

11 

H2 

11 

H012701 

0.803 

(0.012) 

-0.529 

(0.011) 

0.214 

(0.004) 

H2 

13 

H2 

13 

B012801 

1.581 

(0.020) 

-0.471 

(0.012) 

0.255 

(0.004) 

H2 

14 

H2 

14 

B012g01 

2.293 

(0.035) 

-0. 170 

(0.012) 

0.210 

(0. 003) 

H2 

15 

H2 

15 

B013001 

1.421 

(0.013) 

-0.954 

(0.014) 

0.214 

(0. 005) 

H3 

1 

H3 

1 

B013101 

0.885 

(0.012) 

-0.908 

(0.016) 

0.322 

(0. 005) 

H3 

2 

H3 

2 

B013201 

1.  f57 

(0.013) 

-0.iJ80 

(0.013) 

0. 146 

(0.005) 

H3 

3 

H3 

3 

-— 

B013301 

0.860 

(0.013) 

-0.242 

(0.009) 

0.183 

CO. 004) 

H3 

4 

B3 

4 

B013^01 

1.413 

(0.025) 

-0. 127 

(0.011) 

0.279 

(0^003) 

H3 

5 

H3 

5 

B913501 

0.868 

(0.011) 

-0.580 

(0. Oil) 

0. 170 

(0. 005) 

H3 

6 

H3 

6 

BO 13 502 

i  174 

(0.012) 

-1.055 

(0. 014) 

0.216 

(0. 006) 

H3 

7 

H3 

7 

H013701 

0.426 

(O.OOG) 

-1.059 

(0. 023) 

0.220 

(0. 006) 

B3 

8 

H3 

8 

B013801 

1.434 

(0.026) 

-0.030 

(0.011) 

0..'J67 

(0. 003) 

H3 

9 

H3 

9 

B013g01 

1.342 

(0.016) 

-0.590 

(0.012) 

0.220 

(0.004 ) 

H3 

10 

H3 

10 

BOUOOl 

1.609 

(0.022) 

-0.248 

(0.011) 

0. 156 

(0. 003) 

H3 

11 

H3 

11 

BOUXOl 

1.637 

(0.017) 

-0.666 

(0.013) 

0.20S 

v0.004) 

H3 

12 

H3 

12 

BOU201 

1.763 

(0.033) 

0. 132 

(0.013) 

0.248 

(0. 003) 

H3 

13 

h3 

13 

B014301 

1.635 

(0.028) 

0.478 

(0.017) 

0.2'!6 

( 0 . C03 ) 

H3 

14 

H3 

1*1 

B014401 

1.666 

(0.023) 

-0. 198 

(0.010) 

0. 139 

10.  OD'3) 

H3 

15 

H3 

15 

B014S01 

1.867 

(0.016) 

-1. 167 

(0.018) 

0.385 

(0. 006) 

H4 

1 

H4 

1 

B014601 

2.344 

(0.018) 

-1. 123 

(0.010) 

0.35S 

(0. 006) 

H4 

2 

H4 

2 

B014701 

1.277 

(0.010) 

-1.587 

(0.017) 

0. 163 

(0.009) 

H4 

5 

H4 

5 

B014702 

0.902 

(0.009) 

-1.613 

(0.018) 

0.122 

(0.008) 

H4 

6 

H4 

6 

B014703 

1.590 

(0.022) 

-0  301 

(0.011) 

0. 107 

(0.004) 

H4 

7 

H4 

7 

B015001 

0.904 

(0.010) 

-0  831 

(0.012) 

0.004 

(0.005) 

H4 

3 

H4 

3 

BOISIOI 

0.785 

(0.014) 

-0.030 

(0.009) 

0. 199 

(0. 004) 

B4 

4 

H4 

4 

BO 15201 

1.356 

(0.022) 

-0. 383 

(0.013) 

0. 330 

an 

a 
O 

n*l 

a 
O 

B01S301 

O.SOl 

(0.010) 

-0.972 

(0.014) 

0.176 

(0.006) 

B4 

9 

H4 

9 

H01S401 

1.283 

(0.014) 

-0.544 

(0.010) 

0.118 

(0.004) 

H4 

10 

H4 

10 

BO 15501 

2.399 

(0.044) 

0.258 

(0.018) 

0.323 

(0.003) 

B4 

11 

H4 

11 

B015601 

1.046 

(0.021) 

0.236 

(0.012) 

0.269 

(0.003) 

B4 

12 

H4 

12 

B015701 

1.040 

(0.013) 

-0.514 

lO.OlO) 

0.130 

(0.004) 

H4 

13 

H4 

13 

B015801 

1.023 

(0.019) 

0.506 

(O.Oi^) 

0.174 

(0.003) 

H4 

14 

H4 

14 

B015g01 

1.613 

(0.028) 

0.038 

(0.012) 

0.245 

(0.003) 

H4 

15 

H4 

15 

B016001 

0.6S'5 

(0.017) 

-1.459 

(0.040) 

0.221 

(0.012) 

H2 

16 

BOieiOl 

2.267 

(0.051) 

-0.368 

(0.018) 

0.227 

(0.007) 

H2 

17 

B016201 

0.456 

(0.015) 

-0.612 

(0.022) 

0.200 

(0.008) 

H2 

18 

B016301 

2.096 

(0.C35) 

0.333 

(0.018) 

0.352 

(0.005) 

H2 

19 

B016401 

1.511 

(0.035) 

-0.279 

(0.015) 

0.279 

(0.007) 

H2 

20 

B016501 

1.922 

(0.042) 

0.113 

(0.016) 

0.357 

(0.005) 

H2 

21 

Boneoi 

1.983 

(0.036) 

0.137 

(0.014) 

0.230 

(0.005) 

H2 

22 

B016701 

1.551 

(0.026) 

0.166 

(0.013) 

0.176 

(0.005) 

H2 

23 

B016801 

2./66 

(0.035) 

0.422 

(0.019) 

0.253 

(0.004) 

H2 

24 

ERIC 


663 


697 


Table  F-6  (continued) 


1988  IRT  Parameters,  U.S.  History  Cross -sectional 

GMHE  4/AGH  9         GRADE  8/A6E  13       GRADE  12/AGE  17 


KAEP  ID 

A 

S.E. 

B 

S  «Ei 

c 

S.E. 

BLOCK  ITEM 

BIOCK 

1TQ\ 

BtX)CK 

ITEH 

B016901 

l.SOO 

(0.022) 

0.695 

(0.018) 

0.209 

(0.004) 

B2 

25 

H017001 

0.468 

(0.023) 

2.198 

(0.105) 

0.222 

(0.004) 

H2 

26 

B017101 

1.120 

(0.023) 

0.500 

(0.018) 

0.245 

(0.005) 

B3 

16 

B017102 

1.529 

(0.038) 

-0.593 

(0.023) 

0.294 

(0.010) 

B3 

17 

H017301 

1.310 

(0.027) 

0.638 

(0.023) 

0.340 

(0.005) 

B3 

18 

H017A01 

1.119 

(0.024) 

0.445 

(0.018) 

0.253 

(0.006) 

B3 

.^9 

H017501 

1.267 

(0.029) 

-0.372 

(0.ni7) 

0.215 

(0.009) 

B3 

20 

H017601 

1.7f7 

(0.031) 

0.359 

(0.018) 

0.234 

(0.005) 

— —         — — 

B3 

21 

H017701 

1.387 

(0.032) 

1.459 

(0.048) 

0.331 

(0.004) 

B3 

22 

B017601 

0.941 

(0.022) 

0.564 

(0.020) 

0.238 

(0.006) 

B3 

23 

H017901 

1.952 

(0.028) 

1.369 

(0.037) 

0. 186 

(0.003) 

B3 

24 

B016001 

2.571 

(0.026) 

1.109 

(0.029) 

0.154 

(0.003) 

B3 

25 

B018101 

1.319 

(0.023) 

1.219 

(0.030) 

0.186 

(0.004) 

B3 

26 

B016201 

0.818 

(0.020) 

-0.289 

(0.014) 

0. 152 

(0.008) 

B4 

16 

E016301 

1.724 

(0.036) 

0.075 

(0.015) 

0. 191 

(0.006) 

B4 

3"* 

B016401 

1.890 

(0.035) 

0 .  Z66 

(0.018) 

0.271 

(0.006) 

B4 

18 

B016S01 

K016 

(0.025) 

1. 112 

(0.034) 

0.295 

(0.005) 

H4 

19 

B018601 

1  258 

(0.025) 

0.567 

(0.019) 

0.235 

(0.005) 

— —         — — 

H4 

20 

B018701 

0.923 

(0.022) 

0.513 

'  018) 

0.203 

(0.0G6) 

H4 

21 

Boiaeoi 

2.143 

(0.027) 

0.7CJ 

i.022) 

0.207 

(0.004) 

H4 

22 

B018S01 

2.108 

(0. 026) 

0.772 

(0.022) 

0. 198 

(0.004) 

B4 

23 

B019001 

1.139 

(0.023) 

0.  955 

(0.025) 

0. 188 

(0.004) 

H4 

24 

B019101 

1.373 

(0.026) 

1.0*^  ' 

(0.030) 

0.288 

(0.004) 

B4 

21$ 

B019201 

2.390 

(0.025) 

1.1 

(0.C28) 

0. 122 

(0.003) 

25 

8019301 

0.507 

(0.012) 

-1.7v  * 

(0.043) 

0.246 

(0.009) 

R>> 

1 

B6 

1 

B019401 

1.048 

(0.013) 

-0.026 

(0.008) 

0. 181 

(0.005) 

F16 

2 

H6 

2 

B019S01 

1.206 

(0.017) 

-0.115 

(0.009) 

0. 243 

(0.005) 

H6 

3 

H6 

3 

B019601 

1.066 

(0.013) 

0.922 

(0.016) 

0.452 

(0.003) 

B6 

4 

H6 

4 

B019701 

1.007 

(0.017) 

-0.564 

(0.013) 

0. 229 

(0.007) 

B6 

uc 
nb 

< 

B019601 

0.693 

(0.012) 

-0. 112 

(0.009) 

0. 287 

(0.005) 

B6 

b 

uc 
nb 

b 

B019901 

1.060 

(0.020) 

-1.263 

(0.028) 

0.259 

(0.011) 

H6 

10 

uc 
nb 

1  n 

B01990Z 

0. 735 

(0.012) 

-0.850 

(0.016) 

0. 162 

(0.008) 

B6 

1 1 

nc 
nb 

B020101 

1. 121 

(0.014) 

0. 319 

(0.011) 

0.  L  46 

(0.004 ) 

H6 

8 

uc 
nb 

0 

B020201 

1. 959 

(r  022) 

0.  224 

(0.011) 

0.276 

(0.004) 

B6 

1 

nc 
nb 

/ 

B020301 

1. 657 

(0.013) 

0. 616 

(0.014) 

0. 350 

(0.003) 

nb 

1 0 

tic 
nb 

1 9 
kit 

B020401 

1.327 

(0.010) 

0.  875 

(0.012) 

0. 210 

(0.003) 

no 

n 

uc 
no 

0 

9 

B020501 

1.414 

(0. 014) 

0.517 

(0.012) 

0. 357 

(0.004 ) 

nb 

1 1 

tic 

IJ 

B020601 

0.751 

(0. 010) 

0. 087 

(0.008) 

0 . 217 

(0.005) 

H6 

14 

tic 
nb 

1  L 

Xh 

B020701 

1.081 

(0. oil) 

0.  572 

(0.010) 

0 . 202 

( 0. 004  ) 

no 

Xj 

no 

ID 

B020801 

1.500 

(0 .012) 

0. 436 

(0.009) 

0. 118 

(0.003) 

nb 

lb 

uc 
nb 

1  C 
ID 

B020901 

l.v  '0 

(0.014) 

0.  426 

(0.010) 

0. 181 

(0.004 ) 

nc 

X  1 

NC 

no 

1/ 

B021001 

1.866 

(0. 014) 

0 ,638 

(0.013) 

0. 299 

(0.003) 

H6 

tic 
nb 

lb 

B021101 

1.376 

(0.011) 

0.  627 

(0.010) 

0. 169 

(0.003) 

tic 
Ub 

1  Q 

X9 

ib 

1Q 
I9 

B021201 

0.879 

(0.011) 

0.  589 

(0.012) 

0. 349 

(0.004) 

H6 

20 

tic 
nb 

Oft 

B021301 

1.862 

(0.013) 

0. 690 

(0.011) 

0. 198 

(0.003) 

H6 

0 1 

Zl 

nc 
nb 

Zl 

1.  3oj 

/n  nin\ 
\ u. uxu/ 

no 

22 

H6 

22 

B021S01 

1.022 

(0.008) 

1.337 

(0.014) 

0.115 

(0.002) 

H6 

23 

H6 

23 

B021601 

1.390 

(O.Oll) 

1.077 

(0.015) 

0.2Bg 

(0.003) 

B6 

24 

H6 

24 

B021701 

2.60e 

(0.016) 

1.713 

(0.028) 

0.29i 

(0.002) 

H6 

25 

H6 

25 

B021801 

1.705 

(0.012) 

1.496 

(0.020) 

0,287 

(0.003) 

H6 

26 

H6 

26 

B021901 

2.025 

(0.012) 

1.186 

(0.016) 

0.255 

(0.003) 

Hb 

27 

H6 

27 

B022001 

1.657 

(0.013) 

1.886 

(0.026) 

0.225 

(0.002) 

B6 

78 

B6 

28 

B022101 

1.157 

(0.015) 

0.102 

(0.010) 

0.233 

(0.005) 

B7 

1 

B7 

1 

B022201 

0.647 

(0.013) 

-1.047 

(0.024) 

0.200 

(0.009) 

H7 

2 

H7 

2 

B022301 

0.896 

(0.011) 

0.674 

(0.012) 

0.179 

(0.004) 

3 

H7 

3 

B022501 

1.479 

(0.017) 

0.244 

(0.011) 

0.224 

(0.005) 

ti7 

5 

H7 

5 

B022601 

1.379 

(0.021) 

0.006 

(0.012) 

0.303 

(0.006) 

B7 

6 

H7 

6 

B022701 

0.981 

(0.011) 

0.576 

(0.011) 

0.167 

(0.004) 

B7 

7 

H7 

7 

B022801 

1.064 

(0.017) 

0.003 

(0.011) 

0.370 

(0.006) 

H7 

8 

B7 

8 

B022802 

1.082 

(0.013) 

0.665 

(0.014) 

0.352 

(0.004) 

B7 

9 

B7 

9 

B022901 

1.861 

(0.014) 

0.712 

(0.012) 

0.139 

(0.003) 

H7 

10 

H7 

10 

B023101 

2.186 

(0.015) 

0.869 

(0.016) 

0.248 

(0.003) 

B7 

11 

B7 

11 

B023201 

0.936 

(0.012) 

0.308 

(0.010) 

0.193 

(0.005) 

B7 

12 

B7 

12 

B023301 

1.065 

(0.013) 

0.617 

(0.013) 

0.285 

no4) 

B7 

13 

B7 

13 

Table  F-6  (cor-^inued) 


1988  IRT  Parameters,  U.S.  History  Cross-sectional 


GRADE  4/AGE  9         GRADE  8/AGE  13       GRADE  12/AGE  17 


KAEP  ID 

S.E. 

B 

S.E. 

C 

S.E. 

BUXnC  ITEM 

BLOCK 

ITEM 

BLOCK 

ITDj 

B023401 

0.799 

(0.011) 

0.677 

(0.014) 

0.253 

(0.004) 

H7 

14 

B7 

14 

B023S01 

1.351 

(0.012) 

o.aio 

(0.013) 

0.194 

(0.004) 

— 

H7 

15 

B7 

15 

H02C501 

0.766 

(0.011) 

0.547 

(0.012) 

0.194 

(0.C05) 

— 

H7 

16 

B7 

16 

B023y01 

1.843 

(0.012) 

1.101 

(0.016) 

0.204 

(0.003) 

— 

B7 

17 

B7 

17 

H023801 

2.350 

(0.016) 

0.793 

(0.015) 

0.221 

(0.003) 

H7 

18 

B7 

18 

B023901 

1.500 

(0.013) 

0.804 

(0.014) 

0.265 

(0.004) 

-- 

B7 

19 

B7 

19 

H024001 

2.335 

(0.016) 

1.683 

(0  028) 

0.252 

(0.003) 

— 

H7 

20 

H7 

20 

B024101 

1.069 

(0.010) 

0.843 

(0.012) 

0.079 

(0.003) 

H7 

21 

H7 

21 

B024201 

3. 021 

(0.016) 

1.390 

(0.024) 

0.238 

(0.003) 

— 

B7 

22 

E7 

22 

B024301 

1.262 

(0.013) 

1.595 

(0.023) 

0.221 

(0.003) 

— 

H7 

23 

B7 

23 

B024401 

1.776 

(0.015) 

O.BOO 

(0.015) 

0.259 

(0.004) 

— 

B7 

24 

B7 

24 

B024S01 

1.669 

(0.013) 

1.103 

(0.017) 

0.250 

(0.003) 

— 

B7 

25 

B7 

25 

B024601 

1.775 

(0.013) 

1.687 

(0.024) 

0.182 

(0.003) 

B7 

26 

H7 

26 

B02470'4' 

1.622 

(0.014) 

1.682 

(0.026) 

0.258 

(O.0O3) 

— 

B7 

27 

B7 

27 

H024801 

3.042 

(0.021) 

1.942 

(0.041) 

0.235 

(0.002) 

H7 

28 

H7 

28 

B024901 

1.194 

(0.031) 

•0.153 

(0.016) 

0.296 

(0.008) 

— 

R8 

1 

— 

— 

B02SOO3 

1.018 

(0.013) 

1.776 

(0.028) 

0.168 

(0.004) 

— 

— 

H8 

1 

B02S101 

0.915 

(0.012) 

0.439 

(0.013) 

0.218 

(0.007) 

— 

— 

B2 

3 

B02S201 

1.866 

(0.014) 

1.311 

(0.022) 

0.349 

(0.004) 

— 

— 

— 

B2 

4 

H02S301 

1.247 

(0.012) 

1.050 

(0.017) 

0.275 

(0,005) 

— 

- 

— 

B2 

5 

B02S401 

1.303 

(0.012) 

1.641 

(0.024) 

0.280 

(0.004) 

-- 

hZ 

6 

B02SS01 

0.891 

(0.015) 

0.170 

(0.013) 

0.268 

(0.008) 

-- 

H2 

7 

B02SS02 

0.955 

(0.013) 

0.306 

(0.013) 

0.220 

(0.007) 

H2 

8 

B02S701 

1.115 

(0.013) 

1.344 

(0.023) 

0.385 

(0.005) 

-- 

-- 

H2 

9 

B02S801 

1.006 

(0.011) 

0.744 

(0.014) 

0.164 

(0.006) 

— 

-- 

H2 

10 

B02S901 

1.389 

(0.012.' 

0.934 

(0.015) 

0.195 

(0.005) 

-- 

H2 

11 

B026001 

0.910 

(0.010) 

1.020 

(0.016) 

0.178 

(0.005) 

— 

H2 

12 

H026101 

1.168 

(0.012) 

1.203 

(0.019) 

0.285 

(0.005) 

H2 

13 

B026201 

1.188 

(0.013) 

0.479 

(0.013) 

0.172 

(0.007) 

14 

B026301 

1.218 

(0.013) 

0.617 

(0.013) 

0.150 

(0.006) 

15 

H026401 

1.316 

(0.014) 

0.760 

(0.016) 

0.314 

(0.006) 

B2 

16 

B026S01 

1.337 

(0.012) 

1.421 

(0.020) 

0.250 

(0.004) 

H2 

17 

B026601 

1.224 

(0.012) 

1.021 

(0.016) 

0.218 

(0.005) 

H2 

18 

B025701 

C.977 

(0.012) 

1.4Q0 

(0.022) 

0.280 

(0.005) 

B2 

20 

B026801 

0.744 

(0.010) 

0.&i3 

(0.015) 

0.1.^6 

(0.006) 

H2 

19 

B026901 

1.001 

(0.011) 

0.7  5 

(0.013) 

0.130 

(0.006) 

H2 

21 

H027OO1 

1.223 

(O.Ull) 

1.257 

(0.018) 

0.215 

(0.005) 

H2 

22 

H027101 

2.452 

(0.014) 

1.717 

(0.026) 

0.196 

(0.003) 

H2 

23 

B027201 

1.205 

(0.012) 

1.592 

(0.023) 

0.268 

(0.004) 

H2 

24 

B027301 

1.820 

•0.014) 

1.695 

(0.026) 

0.281 

(0.004) 

H2 

25 

B027401 

1.590 

(O.OIA) 

1.924 

(0.029) 

0.202 

(0.003) 

H2 

26 

B027S01 

2.060 

(0.015) 

1.835 

(0.029) 

0.250 

(0.004) 

H2 

27 

B027601 

2.026 

(0.019) 

2.105 

(0.039) 

0.226 

(0.003) 

H2 

28 

B028001 

1.379 

(0.022) 

0.148 

(0.017) 

0.392 

(0.008) 

H2 

1 

H038101 

0.814 

(0.012) 

0.372 

(0.012) 

0.186 

(0.007) 

B2 

2 

ERIC 


665 


Table  F-7 


1988  IRT  Parameters,  Geography  C^oss- sectional 


HAEP  IP 

GOOOlOl 
G000201 
G000301 
G000401 
GOOO^Ol 
G000601 
G000701 
G000801 
G000901 
GOOlOOl 
GOOllOl 
G001201 
G001301 
G001302 
GOOUOl 
GOOlSOl 
G001601 
G001701 
G001801 
GOOlOOl 
G002001 
G002101 
G002201 
G002301 
G002401 
G002S01 
G002601 
G002602 
G002701 
G002801 
G002001 
G003001 
G003101 
G003201 
GC03301 
CC03401 
GC03S01 
GC03S02 
G003601 
G003701 
G003801 
G003901 
G004001 
G004101 
G00420I 
G004301 
G004401 
G00U02 
G004403 
G004404 
GOQASOl 
G004601 
G004701 
G004801 
G004901 
GOOSOOl 
G00S002 
G005iui 
G00S102 
G00S201 
G00S202 
G30S203 
G00S301 
0003401 


0.823 
0.622 
0.567 
0.722 
1.109 
0.870 
0.GS3 
0.938 
0.380 
0.383 
0.9G2 
0.:S92 
1.685 
1.060 
0.786 
1.358 
1.084 
1.050 
0.710 
0.661 
1.704 
1.252 
1.452 
1.296 
0.969 
0.886 
1.001 
0.479 
K'  908 
0.800 
1.121 
0.658 
1.051 
1.050 
1.370 
0.404 
0.858 
1.046 
1.367 
0.704 
1.225 
0.888 
1.164 
0.713 
0.940 
0.984 
0.807 
0.934 
1.772 
1.386 
0.3«:) 
1.741 
1.166 
1.191 
1.041 
1.122 
1.765 
1.066 
1.292 
0.614 
0.724 
1.543 
1.446 
0.808 


S.E. 

(0.030) 
(0.025) 
(0.023) 
(0.028) 
(0.035) 
(0.027) 
(0.032) 
(0.029) 
(0.021) 
(0.019) 
(0.032) 
(0.021) 
(0.041) 
(0.031) 
(0.031) 
(0.039) 
(0.030) 
(0.031) 
(0.022) 
(0.029) 
(0.035) 
(0.033) 
(0.030) 
(0.030) 
(0.035) 
(0.033) 
(0.034) 
(0.024) 
(0.035) 
(0.031) 
(0.041) 
(0.029) 
(0.034) 
(0.036) 
(0.046) 
(0.022) 
(0.032) 
(0.029) 
(0.039) 
(0.029> 
(0.03; , 
(0.034) 
(0.035) 
(0.032) 
(0.029) 
(0.032) 
(0.032) 
(0.033) 
(0.037) 
(0.034) 
(0.031) 
(0.087) 
(0.035) 
(0.042) 
(0.036) 
(O.035) 
(0.053) 
(0.C32) 
(0.045) 
(0.028) 
(0.024) 
(0.033) 
(0.041) 
(0.027) 


-0,168 
-0.271 
-1.238 
-0.013 
-0.578 
-0.783 
-0.070 
0.295 
0.471 
0.130 
1.552 
-0.392 
0.834 
0.538 
0.43» 
0.47a 
0.576 
0.741 
0.206 
0.490 
1.429 
1.582 
1.661 
1.278 
1.952 
-0.663 
0.261 
0.484 
-0.432 
-0.507 
-0.340 
-0.168 
-0.380 
0.846 
0.451 
-0.303 
0.568 
1.094 
0.549 
0.964 
0.307 
0.709 
0.880 
1.221 
1.320 
0.585 


.138 
.716 
.171 
.428 
,117 
.272 
.003 
0.121 
0.156 
-0.228 
-0,778 
0.317 
-0,216 
-0.100 
0.315 
0.735 
0.034 
-0.27 


S.E. 

(0.022) 
(0.022) 
(0.055) 
(0.020) 
(0.028> 
(0.031) 
(0.021) 
(0.022) 
(0.032) 
(0.019) 
(0.062) 
(0.022) 
(0.043) 
(0.027) 
(0.029) 
(0.028) 
(0.029) 
(0.033) 
(0.018) 
(0.031) 
(0.053) 
(0.058) 
(C  0';7) 
(O.044) 
(0.0S3) 
(0.034) 
(0.026) 
^0.033) 
(0.028) 
(0.030) 
(0.028) 
(0.023) 
(0.025) 
(0.043) 
(0.037) 
(0.026) 
(0.032) 
(0.042) 
(0.035) 
(0.048) 
(0.026) 
(0.039) 
(0.042) 
(0.062) 
(0.049) 
(0.032) 
(0.049) 
(0.037) 
(0.048) 
(0.054) 
(0.172) 
(0.194) 
(0  021) 
(0.027) 
(0.024) 
(0.022) 
CO. 041) 
(0.024) 
(0.025) 
(0.020) 
(0.021) 
:0.032) 
(0.021) 
(0.020) 


0.291 
0.286 
0.300 
0.3D:* 
0.187 
0.1&5 
0.239 
0.201 
G.280 
0.231 
0. 2oS 
0.180 
0.422 
0.223 
0.354 
0.259 
0.259 
0.275 
0.114 
0.336 
0.263 
0.245 
0.277 
0.209 

o.2n 

0.289 
0.256 
0.244 
0.229 
0,244 
0.238 
0.262 
0.131 
0.325 
0.390 
0,238 
0.233 
0,158 
0,312 
0.233 
0.170 
0,301 
0,296 
0,262 
0.142 
0.213 
0.251 
0.250 
0.221 
0.168 
0.270 
0.286 
0.186 
0.391 
.339 
.187 
.298 
'.47 
.275 
.224 
.195 
.231 
0.141 
0.172 


S.E. 

0.010) 
0.010) 
0.015) 
0.C09) 
0.012} 
0.013) 
0.009) 
U.008) 
0.009) 
0.010; 
0.006) 
0.011) 
0.006) 
0.007) 
0.008) 
0.007) 
0.CO7) 
0.007) 
0.008) 
0,008) 
0.005) 
0,C05) 
0.005) 
0,006) 
0.005) 
0.013) 
0.009) 
0.010) 
0.012) 
0.013) 
0.032) 
0.011) 
012) 
008) 
008) 
012) 
008) 
006) 
0.008) 
0.008) 
0.008) 
0.008) 
0.007) 
U.008) 
0.006) 
0.008) 
0.018) 
0.008) 
0.006) 
0.006) 
0.008) 
0.006) 
0.008) 
0.008) 
0.008) 
0.010) 
0.013) 
0.008) 
O.CIO) 
0.010) 
0.008) 
0.006) 
0.008) 
0.010) 


GRADE  4/AGV  9 
ByOCK  ITEM 


GRADE  8/AGE  13 
BLOCK  ITEM 


GRADE  12/AGE  17 
BLOCK  ITEM 


G2 

1 

G2 

2 

G2 

3 

G2 

4 

G 

5 

G' 

6 

7 

G2 

8 

G2 

9 

G2 

10 

G2 

11 

G2 

12 

G2 

14 

G2 

15 

G2 

16 

G2 

17 

G2 

13 

G2 

le 

G2 

19 

G2 

22 

G2 

20 

G2 

21 

G2 

23 

G2 

24 

G2 

2.^ 

G2 

Z6 

G3 

1 

G3 

2 

G3 

3 

G3 

4 

G3 

5 

G3 

6 

G3 

7 

G3 

8 

G3 

9 

G3 

10 

G3 

11 

G3 

12 

G3 

15 

G3 

14 

G3 

15 

G3 

16 

C3 

17 

G3 

18 

G3 

10 

G3 

20 

G3 

21 

G3 

22 

G3 

23 

G3 

24 

G3 

25 

G3 

26 

G4 

1 

G4 

2 

G4 

3 

G^, 

4 

G4 

5 

G4 

6 

G4 

7 

GA 

8 

9 

G4 

10 

G4 

11 

12 

ERIC 


666 


7U0 


Table  F-7  (continued) 
1988  IRT  Parameters,  Geography  Cross -sectional 


KAEP  ID 

GRADE  4/AGE  9 

GRADE  8/AGE  13 

GRADE 

A 

s.s. 

B 

S.E. 

C 

S.E. 

BLOCK  ITEM 

BLOCK  ITEM 

BLOCK 

ITEM 

G005501 

1.517 

(0.046) 

-0.031 

(0.023) 

0.191 

(0.008) 

G4 

13 

G005601 

0.490 

(0.022) 

1.115 

(0.0t>4) 

0.206 

(0.007) 

G4 

14 

GO0S701 

0.949 

(0.030) 

0.087 

(0.021) 

0.252 

(0.009) 

G4 

15 

6005801 

1.808 

(0.037) 

0.591 

(0.030) 

0.210 

(0.006) 

G4 

16 

G005901 

1.476 

(0.038) 

0.407 

(0.028) 

0.275 

(0.007) 

G4 

17 

G00800X 

0.798 

(0.027) 

1.286 

(0.051) 

0.229 

(0.006) 

G4 

18 

G006101 

2.311 

(0.042) 

0.688 

(0.035) 

0.250 

(0.006) 

G4 

19 

G006201 

0.276 

(0.025) 

2.294 

(0.213) 

0.322 

(0.007) 

G4 

20 

GOO 6301 

0.344 

(0.018) 

0.356 

(0.025) 

0.185 

(0.010) 

G4 

21 

G006S01 

1.149 

(0.027) 

0.852 

(0.031) 

0.146 

(0.006) 

G4 

22 

G006601 

0.079 

(0.014) 

6.425 

(1.183) 

0.192 

(0.008) 

G4 

23 

GO06701 

1.542 

(0.048) 

0.180 

(0.028) 

0.293 

(0.008) 

G4 

24 

G006801 

0.985 

(0.031) 

1.746 

(0.066) 

0.175 

(0.005) 

G4 

25 

G00690X 

1.071 

(0.027) 

1.037 

(0.037) 

0.176 

(0.006) 

G4 

26 

ERIC 


667 


Table  F-8 


1988  IRT  Parameters,  Mathematics  Trend,  Age  9 


AGE  9 


HAEP  ID 

SJL. 

B 

S.£. 

Q 

S.E. 

ITEM 

K?50fi01 

097 

(0.076) 

-0 

231 

(0 

045) 

n 

919 

(0 

019) 

M2 

lo 

n 
u 

791 

(0.053) 

-0.584 

(0 

.054) 

n 

u 

.  109 

(0 

0?.3) 

M9 

1  A 
1*1 

355 

(0.071) 

0 

566 

(0 

056) 

n 
U 

.  1  JO 

(0 

013) 

M9 

1  <^ 
1 J 

n 

mo 

(0.044) 

"0 

8SC 

(0 

.059) 

n 
U 

1  "SO 

.  1 J9 

(0 

022) 

Ml 

7 
/ 

•i^ww  / 

1 

X 

nni 

(0.048) 

Ox  1 

(0 

054) 

117 

(0 

oil) 

Ml 

o 
O 

K25070'* 

1 

V-JH 

(0 

.064) 

0 

OlD 

(0 

033) 

0 

.123 

(0 

016) 

Ml 

Q 
9 

H250901 

f) 

599 

(0 

"0 

All 

411 

(0 

.040) 

0 

.178 

(0 

019) 

M2 

^  7 
X/ 

ini 
xux 

1 

181 

(0 

072) 

0 

.157 

(0 

010) 

M9 

1 A 
XO 

11250903 

4  <  d  ^  W  V  W  ^ 

0 

970 

(v 

x) 

DOJ 

(0 

.050) 

0 

.109 

(0 

012) 

M2 

1  Q 
X9 

K?51401 

n 

654 

(0 

.u42) 

"0 

265 

(0 

036) 

0 

.151 

(0 

021) 

M9 

XD 

(0 

.131) 

2 

670 

(0 

.372) 

0 

.196 

(0 

009) 

M2 

9'? 

11957101 

n 

839 

(0 

.060) 

1 

(0 

.143) 

0 

.170 

(0 

012) 

Ml 

(0 

.084) 

"0 

533 

(0 

.055) 

0 

.28? 

(0 

020) 

Ml 

Ml 

XX 

/ ou  X 

u 

^OO 

(0 

.038) 

"0 

909 

0 

.063) 

0 

240 

(0 

022) 

M9 

•a 
O 

n 
u 

676 

(0 

.065) 

1. 

029 

(0 

.092) 

0 

.236 

(0 

012) 

Ml 

no 

1  Q 
X9 

XI^Dm^U  X 

n 

509 

(0 

.042) 

"0 

143 

(0 

.037) 

0 

232 

(0 

020) 

M9 

1  9 
X£ 

n^D^^u  X 

0 

441 

(0 

.036) 

"1 

218 

(0 

.105) 

0 

196 

(0 

024) 

M1 

ni 

lU 

i  1  &  U  &  ^  W  X 

n 

594 

(0 

.069) 

0 

928 

(0 

.116) 

0 

300 

(0 

013) 

Ml 

no 

1  o 
lO 

n 

269 

(0 

.031) 

"0 

coo 
boo 

(0 

084) 

0 

.227 

(0 

019) 

Ml 

Ml 

1  Q 
l9 

n 

254 

(0 

062) 

D 

1  CO 
1D9 

(1 

.519) 

0 

.172 

(0 

008: 

Ml 

Ml 

9n 

R?R?401 

Ct&U  V  X 

n 

888 

(0 

063) 

11 

7ni 

(0 

.063) 

0 

.299 

(0 

.022) 

M9 

nin 

UXU 

(0 

.080) 

-n 
u 

(0 

.043) 

0 

.282 

(0 

018) 

M2 

c 

J 

H265401 

582 

(0 

.164) 

(0 

.360) 

0 

.340 

(0 

Oil) 

M1 

Ml 

91 
^X 

11266101 

Q 

542 

(0.052) 

1 

Q1  7 
91/ 

(0 

192) 

O.lo 

(0 

Oil) 

Ml 

nx 

22 

H267001 

Q 

597 

(0 

.045) 

.1 
1 

(0 

.110) 

0 

.249 

(0 

026) 

M3 

16 

H?fi7fi01 

266 

(0.066) 

-0 

611 

(0 

049) 

0 

.156 

(0 

020) 

Ml 

nx 

o 
o 

R267602 

1 

103 

(0.057) 

-0 

074 

(0 

031) 

0 

.104 

(0 

014) 

Ml 

nx 

16 

R266201 

1 

248 

(0 

058) 

1 

026 

(0 

068) 

0 

.201 

(0 

010) 

Ml 

nx 

24 

R269001 

0 

565 

(0 

.087) 

4 

055 

(0 

634) 

0 

.082 

(0 

007) 

M2 

26 

R269101 

0 

540 

(0 

.071) 

2 

970 

(0 

.402) 

0 

238 

(0 

009) 

Ml 

nx 

23 

R270001 

Q 

448 

(0 

030) 

-0 

727 

(0 

.053) 

0 

.000 

(0 

000) 

Ml 

nx 

14 

H270901 

0 

694 

(0 

.037) 

-2 

165 

(0 

098) 

0 

000 

(0 

000) 

Ml 

nx 

H271101 

0 

626 

(0 

034) 

-0 

305 

(0 

028) 

0 

000 

(0 

000) 

M2 

24 

R272101 

Q 

990 

(0 

.096) 

-0 

533 

(0 

071) 

0 

286 

(0 

024) 

MS 

17 

R272102 

Q 

992 

(0 

.062) 

0 

034 

(0 

039) 

0 

173 

(0 

018) 

Ml 

15 

H272301 

0 

946 

(0 

052) 

-1 

947 

(0 

123) 

0 

180 

(0 

040) 

N2 

1 

R272eoi 

0 

576 

(0 

.049) 

-2 

007 

(0 

176) 

0 

180 

(0 

036) 

M3 

15 

R273S01 

0 

744 

(0 

.058) 

-0 

684 

(0 

068) 

0 

.261 

(0.026) 

M2 

6 

H275401 

0 

985 

(0 

.043) 

-0 

478 

(0 

033) 

0 

.000 

(0 

000) 

M2 

7 

H276001 

0 

879 

(0 

.037) 

-0 

975 

(0 

049) 

0 

000 

(0 

000) 

M2 

21 

R276002 

0 

778 

(0.035) 

1 

507 

(0 

.074) 

0 

000 

(0 

000) 

M2 

22 

R276101 

0 

963 

(0  040) 

-0 

756 

(0 

042) 

0 

000 

(0.000) 

Ml 

12 

R276601 

1 

061 

(0.062) 

-1. 

010 

(0 

076) 

0 

170 

(0 

029) 

H2 

2 

H276eoi 

0 

490 

(0 

045) 

-3. 

763 

(0 

353) 

0 

000 

(0.000) 

Ml 

4 

R276e02 

0 

725 

(0 

038) 

-1 

591 

(0 

090) 

0 

000 

(0 

000) 

Ml 

5 

R276e03 

0 

621 

(0.035) 

0. 

147 

(0 

027) 

0 

000 

(0.000) 

Ml 

6 

R277401 

1 

026 

(0 

063) 

-1 

573 

(0 

114) 

0 

177 

(0 

038) 

Ml 

2 

R277501 

0 

842 

(0 

039) 

-0.421 

(0 

031) 

0 

000 

(0.000) 

M2 

8 

H277601 

1 

438 

(0 

049) 

-0 

522 

(0 

037) 

0 

000 

(0 

000) 

M2 

9 

R177602 

1 

267 

(0 

053) 

0. 

172 

(0 

029) 

0 

000 

(0 

000) 

M2 

10 

5277603 

1 

507 

(0 

063) 

-0. 

Oil 

(0 

030) 

0 

000 

(0 

000) 

M2 

11 

H284001 

0. 

961 

(0 

050) 

-0. 

363 

(0 

033) 

0 

000 

(0 

000) 

Ml 

16 

X284002 

0. 

792 

(0 

037) 

2. 

054 

(0 

103) 

0 

000 

(0.000) 

Ml 

17 

H286101 

0. 

814 

(0.039) 

-0. 

521 

(0 

035) 

0 

000 

(0. 

000) 

Ml 

13 

702 


668 


Table  F-9 


1988  IRT  Parameters,  Mathematics  Trend,  Age  13 


AGE  13 


NAEP  ID 

S.E. 

Q 

S.E. 

c 

S.E. 

0. 

688 

(0. 

035) 

-2 . 

717 

(0. 

143) 

0 . 

106 

(0. 

033) 

K2 

14 

1 

1A5 

(0. 

051) 

-Q  ^ 

797 

(0. 

047) 

0 . 

102 

(0. 

018) 

M2 

15 

H250703 

0. 

6Ag 

(0. 

031) 

-2. 

110 

(0. 

106) 

0 , 

110 

(0. 

028) 

M2 

16 

0. 

A23 

(0. 

029) 

"2 . 

565 

(0. 

176) 

0 . 

152 

(0. 

027) 

Ml 

25 

N250902 

1. 

020 

(0. 

049) 

"O . 

349 

(0. 

031) 

0 , 

075 

(0. 

014) 

Ml 

26 

H250903 

0.820 

(0 

039) 

"1  ^ 

510 

(0. 

078) 

0 . 

096 

(0. 

025) 

Ml 

27 

H252001 

1 

A23 

(0 

06A) 

0 . 

832 

(0. 

062) 

0 . 

1  7Q 

XfO 

(0. 

010) 

M2 

40 

N252101 

0 

933 

(0 

056) 

0 . 

623 

(0. 

054) 

0 . 

240 

(0 

013) 

Ml 

4\ 

1 

2A9 

(0.072) 

"0 . 

036 

(0. 

033) 

0 . 

109 

(0 

015) 

Ml 

32 

0 

361 

1  } 

"0 . 

504 

(0. 

050) 

0 . 

27 1 

(0 

016) 

M2 

22 

N254001 

1 

161 

(0 . 

08A  ) 

"0 . 

479 

(0 

047) 

Q 

118 

(0 

017) 

M3 

28 

1 

092 

t  n 

} 

-  ^  ^ 

553 

(0 

089) 

Q 

284 

(0 

030) 

Ml 

16 

0 

7AA 

t  n 

U*l^  } 

413 

(0 

C95) 

Q 

235 

(0 

009) 

Ml 

46 

N2SS701 

1 

317 

(0 

0A4 ) 

268 

(0. 

063) 

n 
u 

(0 

008) 

Ml 

50 

A 

0 

760 

(0 

033  ) 

"  1 . 

(0. 

05"; 

0 

000 

(0 

000) 

M2 

17 

0 

86", 

(0 

069  ) 

0 . 

JOX 

;o 

061) 

Q 

318 

(0 

012) 

M3 

30 

H256801 

1 

05-. 

(0 

069) 

0 . 

841 

(0. 

072) 

Q 

312 

(0 

Oil) 

M3 

32 

N7S7R01 

1.280 

(0 

051)) 

"0 . 

538 

(0. 

035) 

Q 

000 

(0 

000) 

Ml 

35 

1.273 

(0 

055) 

124 

(0 

076) 

Q 

397 

(0 

010) 

Ml 

38 

1 

619 

(0 

078) 

0 

484 

(0 

051) 

Q 

254 

(0 

Oil) 

M2 

31 

N258803 

1 

191 

(0 

044 ) 

2^ 

351 

(0 

068) 

0 

170 

(0 

007) 

M2 

41 

N260101 

11  ^  U  U  X  U  X 

1.299 

(0 

072) 

0 . 

415 

(0 

042) 

0 

160 

(0 

Oil) 

Ml 

43 

0 

833 

(0 

049) 

Oil 

(0 

070) 

Q 

219 

(0 

010) 

Ml 

47 

0 

525 

(0 

052) 

2^ 

619 

(0 

166) 

Q 

219 

(0 

012) 

M2 

38 

N261301 

0 

700 

(0 

046) 

Q 

768 

(0 

062) 

Q 

113 

(0 

012) 

M2 

37 

0 

661 

(0 

056) 

~0 

545 

(0 

055) 

Q 

141 

(0 

020) 

M2 

34 

H261601 

il^U  X  U  U  X 

0 

3AA 

(0 

043) 

903 

(0 

239) 

155 

(0 

012) 

M2 

36 

H261801 

il^U  X  W  X 

0 

679 

(0 

053) 

0 

044 

(0 

033) 

0 

223 

(0 

017) 

M2 

35 

N262201 

0 

520 

(0 

037 ) 

-1 

789 

(0 

132) 

0 

361 

(0 

.023) 

M2 

18 

N262401 

il  ^  U  ^  "V  U  X 

0 

85A 

(0 

,  054  ) 

~0 

556 

(0 

048) 

0 

323 

(0 

017) 

Ml 

28 

N262S01 

0 

360 

(0 

.033) 

-0 

237 

(0 

034) 

0 

348 

(0 

015) 

Ml 

33 

N262S02 

1.216 

(0 

068) 

1 

974 

(0 

151) 

0 

379 

(0 

008) 

Ml 

34 

11263101 

C 

527 

(0 

027) 

-0 

291 

(0 

024) 

0 

000 

(0 

000) 

Ml 

39 

N263A01 

0 

.675 

(0 

.046) 

"2 

751 

(0 

.196) 

0 

257 

(0 

040) 

M2 

12 

K263A02 

0 

.635 

(0 

045) 

-2 

478 

(0 

181) 

0 

263 

(0 

036) 

M2 

13 

H263501 

1 

.389 

(0 

092) 

0 

187 

(0 

036) 

0 

115 

(0 

012) 

M2 

3u 

N26A7CI 

1 

175 

(0 

056) 

0 

867 

(0 

059) 

0 

206 

(C 

..10) 

M2 

03 

N265201 

0 

810 

(0 

.062) 

-1 

548 

(0 

.127) 

0 

339 

(0 

032) 

Ml 

36 

N265202 

0 

6S3 

(0 

074) 

-0 

176 

(0 

.041) 

0 

.339 

(0 

018) 

Ml 

30 

N26S901 

0 

.933 

(0 

.060) 

0 

930 

(0 

079) 

0 

.333 

(0 

012) 

Ml 

40 

H265902 

1 

.077 

(0 

.073) 

1 

170 

(0 

.103) 

0 

.328 

(0 

Oil) 

M3 

31 

N266101 

0 

.8A9 

(0 

065) 

-0 

161 

(0 

.033) 

0 

.292 

(0 

014) 

M3 

27 

N266801 

0 

.559 

(0 

038) 

-1 

108 

(0 

.080) 

0 

248 

(0 

021) 

Ml 

31 

N267201 

11  ^  U  r  ^  U  X 

0 

.776 

(0 

058) 

- 1 

.051 

(0 

.087) 

0 

254 

(0 

026) 

Ml 

23 

II&bU  O  w  U  X 

1 

.012 

(0 

.053) 

0 

382 

(0 

036) 

0 

.  152 

(0 

Oil) 

Ml 

44 

N269101 

0 

.752 

(0 

.04&> 

-0 

384 

(0 

.037) 

0 

.213 

(0 

OIJ) 

M2 

16 

R269901 

0 

.66A 

(0 

.049) 

~c 

274 

(0 

.035) 

0 

288 

(0 

.015) 

M3 

Z9 

N270301 

0 

.A21 

(0 

.031) 

-1 

.596 

(0 

.119) 

0 

126 

(0 

.022) 

M2 

20 

N270302 

1 

.018 

(0 

.047) 

2 

194 

(0 

.118) 

0 

.051 

(0 

.005) 

M2 

21 

N273901 

1 

.786 

(0 

.111) 

0 

..>58 

(0 

.047) 

0 

.184 

(0 

.013) 

Ml 

-^7 

N27A801 

0 

.629 

(0 

.051) 

-0 

.192 

(0 

.036) 

0 

.269 

(0 

.018) 

Ml 

29 

N27S001 

0 

.9A6 

(0 

.040) 

0 

.363 

(0 

.027) 

0 

.000 

(0 

.000) 

Ml 

42 

N27S301 

0.372 

(0 

.028) 

-1 

.728 

(0 

.  x32) 

0 

.147 

(0 

.022) 

M3 

25 

N276801 

0 

.A33 

(0 

.049) 

-4 

.715 

(0 

.  ^*2I 

0 

.000 

(0 

.000) 

Ml 

17 

N276802 

0 

.A93 

(0 

.044) 

-3 

.957 

(0 

.359) 

0 

.000 

(0 

.000) 

Ml 

18 

N276803 

0 

.435 

(0 

.033) 

-1 

.927 

(0 

.148) 

0 

.000 

(0 

.000) 

Ml 

19 

N277401 

0 

.778 

(G 

.056) 

-2 

.903 

(0 

.220) 

0 

.145 

(0 

.042) 

M2 

6 

R277601 

0 

.856 

(0 

.036) 

-2 

.504 

(0 

.113) 

0 

.000 

(0 

.000) 

Ml 

20 

H277602 

0 

.62A 

(0 

.030) 

-1 

.8u^ 

(0 

.095) 

0 

.000 

(0 

.000) 

M^ 

21 

H277603 

0 

.617 

(0 

.031) 

-2 

.287 

(0 

0 

.000 

(0 

.COO) 

Mi 

22 

N277g01 

0 

.591 

(C 

033) 

-3 

.506 

(0 

199) 

0 

.000 

(0 

.000) 

M2 

9 

N277g02 

0 

.688 

(0 

.036) 

-3 

.301 

(0 

.178) 

0 

.000 

(0 

.000) 

hJ. 

10 

N277g03 

0 

.573 

(0 

.030) 

-2 

.859 

(0 

.154) 

0 

.000 

^0 

.000) 

M2 

11 

(1278901 

1 

.559 

(0 

.086} 

0 

.415 

(0 

.051) 

0 

.212 

(0 

.013) 

M2 

32 

669 


703 


Table  F-9  (continued) 


1988  IRT  Parameters,  Mathematics  Trend,  Age  13 


AGE  13 


HAEP  ID 

A 

B 

S.E. 

C 

S.E. 

ITEM 

M278902 

0.720 

(0.051) 

1.338 

(0.107) 

0.216 

(0.012) 

M2 

29 

M278903 

1.338 

(0.058) 

1.066 

(0.073) 

0.169 

(0.010) 

M2 

42 

M278904 

1.315 

(0.057) 

1.487 

(0.097) 

0.194 

(0.010) 

Ml 

49 

M281401 

0.728 

(0.050) 

1.711 

(0.127) 

0.106 

(0.009) 

M2 

39 

M281901 

0.925 

(0.040) 

-2.181 

(0.105) 

0.146 

(0.034) 

Ml 

15 

M282201 

1.063 

(0.058) 

0.576 

(0.051) 

0.343 

(0.011) 

28 

M282202 

0.936 

(0.066) 

-0.458 

(0.045) 

0.255 

(0.017) 

MS 

26 

M283101 

1.57P 

(0.049) 

1.554 

(0.080) 

0.148 

(0.006) 

Ml 

51 

H286201 

0.89.. 

(0.051) 

-0.892 

(0.061) 

0.243 

(0.021) 

Ml 

24 

M286301 

1.189 

(0.050) 

0.660 

(0.046) 

0.205 

(0.010) 

Ml 

45 

N286S01 

1.256 

(0.042) 

1.161 

(0.05B) 

0.141 

(0.008) 

Ml 

48 

M286S02 

1.671 

(0.054) 

1.171 

(0.068) 

0.160 

(O.OOG) 

M2 

43 

M286601 

1.69S 

(0.059) 

-0.194 

(0.029^ 

0.000 

(0.000) 

M2 

23 

K286602 

1.363 

(0.051) 

-0.247 

(0.027) 

0.000 

(0.000) 

24 

M2O6603 

1.494 

(0.050) 

0.405 

(0.030) 

0.000 

(0.000) 

rt2 

25 

Table  F-10 


1988  IRT  Parameters,  Mathematics  Trend,  Age  17 


AGE  17 


A 

S.E. 

S.E. 

c 

1. 

166 

ft    n  i  n 
0  .  9*9 

(0. 

0*1. 

O.OPO 

/ft 
(0 . 

n  nn  \ 
UUU  J 

Ml 

Ml 

N9^i 7n 1 

0 

892 

(0 . 

Unb } 

n  nn  c 

(0. 

029) 

0. 

1*7 

(0 

015) 

M2 

1.6*7 

( 0 

083 ) 

0  .011 

(0.0*J> 

0 . 

259 

(0 

m  0  \ 

Ml 

OQ 

00 

0.930 

(0 . 

057) 

1 . 032 

(O.OC 

0. 

*79 

(0 . 

Oil ) 

Ml 

*0 

MO  conn  4 

1. 

168 

(0 

0*fl) 

0.915 

(0. 

060) 

0 

322 

(0 . 

Oil ) 

Ml 

A  1 

Moconnx 

1. 

576 

(0 

062) 

0  .700 

(0. 

058) 

0. 

359 

(0 

Oil) 

^^l 

AO 
H^ 

MOCAnm 

0 

923 

(0 

n& &  ^ 

(0. 

050) 

0 

186 

(0 

020 ) 

M2 

0  1 

Mocxon 1 

1. 

035 

(0 

051 ) 

0 .08* 

(0. 

033) 

0 

258 

(0 

013) 

Ml 

00 

Mocxcni 

1. 

300 

(0 

0*9) 

~1 .815 

(0 

089) 

0 

237 

(0 

no7  \ 

M2 

13 

MO  cx  cno 

1. 

575 

(0 

070) 

"0 .02* 

(0 

036) 

0 

2?.l 

(0 

012) 

Ml 

07 

C.I 

MO  cc^  n 1 

1 

539 

( 0 

n  CO  ^ 

1 .503 

(0 

086) 

0 

219 

(0 

009) 

MO 

AC 

<ib 

MO  cccn 1 

0 

808 

(0 

05*  ) 

0 . 668 

(0 

059) 

0 

232 

(0 

013) 

MO 

00 

MO  cccn 1 

1 

2*8 

t  n 
\v 

n  CO  ^ 

1  C7C 

1 . 3/b 

(0 

107) 

0 

332 

n  1  1  ^ 
Ul  1  ^ 

MO 

A  C 

1 

451 

t  n 

Ubl  ^ 

"O . 609 

(0.0*5) 

0. 

201 

n  1  fl  ^ 
Ulo^ 

Ml 

Ml 

00 
JZ 

0 

679 

\  u 

n')n  ^ 
U*}U } 

1 .  bbo 

(0 

080 ) 

0 . 

000 

nnn  ^ 
UUU  J 

MO 

A  Q 
*t  9 

Moccnn 1 

1 

055 

(0 

ncQ  \ 
Ubo^ 

0 . 066 

(0 

no7  ^ 
UZ/  } 

0 

000 

(0 

000 ) 

MO 

OA 

MOCC1 n 1 

1 

003 

t  n 

noQ  ^ 

—  1   A  n7 
1  .'<iU/ 

t  n 

n ci  \ 

0 

000 

n  An  ^ 
UUU ) 

Ml 

Ml 

1  c 
ID 

MOCCQn 1 

1 

300 

(0 

n  CO  \ 

~0 .268 

t  n 

\joO} 

0. 

265 

(0 

n  1  c  \ 

Ml 

Ml 

OR 

Jb 

>ioc7i  n  1 

0 

579 

U  } 

1    Q  o 
1  . 03  J 

1 Q 1  ^ 
101 } 

0 

25* 

t  n 

f  1 1  ^ 
I  llj 

MO 

oc 

Mocfifin  1 

1 

90* 

(0 

110) 

n  oi  R 
u .  ^ib 

t  n 

0 

28* 

(0 

012) 

MO 

on 

JO 

Hocflono 

1 

728 

1 1\ 
\v 

089) 

—  n  1  7  c 
U .  1/ J 

t  n 

nAO  ^ 

0 

256 

(0 

01*) 

Ml 

Ml 

oc 
Zb 

0.992 

(0 

0*5) 

0  .250 

t  n 

Moo ) 

0 

222 

(0 

Oil) 

Ml 

Ml 

07 

01 

MOCADnA 

0 

CQO 

bo^ 

(0 

037) 

_  4  QCO 

1  n  c  ^ 

0 

25* 

(0 

029) 

Ml 

Ml 

18 

MO<nnn 1 

1 

188 

t  n 

n&  c  ^ 

~0 .218 

t  n 

n9c  \ 

0 

000 

t  n 

000 ) 

M2 

0 1 
Jl 

1 

235 

UDD  ^ 

-n  ooc 

f  n 

ni7  ^ 
UJ/  ) 

u 

289 

t  n 

n  1  A  ^ 

Ml 

ni 

zo 

HORni ni 

1 

*60 

t  n 

n  cc  ^ 

U  .  9/  J 

vO 

05*  ) 

0 

195 

t  n 

noo  \ 

MO 

nz 

on 
ZU 

HORncni 

n^DUDUl 

1 

^99 

-  1    1  c 

1 .  Ub 

/  n 

n AO  ^ 

0 

000 

t  n 

nnn  ^ 
UUU  J 

Ml 

ni 

1  R 

ID 

Hocnflm 
nZbUoUl 

1 

301 

t  n 

0 . 388 

t  n 

0 

000 

t  n 

nnn  ^ 
UUU  J 

HO 

Mocnon 1 

2 

210 

t  n 

U .  uob 

n  A  c  \ 

0 

157 

t  n 

m  0  ^ 

Ml 

ni 

00 

MOC1 nni 

0 

806 

t  n 

-n  7*^^ 

\i  .  i  OH 

(0 

n  CO  \ 
\)o^} 

0 

216 

f  n 

noo  ^ 

MO 

*iU 

7io  R 1 0  n  1 

0 

510 

k  n 

UJi  ^ 

-  .  CI  n 

X .  310 

(0 

nQ7  \ 
uy  /  J 

0 

01  c 

-  n 

U^v  ) 

M'} 

nz 

Moei oni 

0 

581 

UJi  ^ 

—  1  OQU 

n"'A  ^ 

0 

153 

t  n 

M9 

nz 

MOC1  cm 

0 

775 

(0 

036) 

-o  007 

(0 

113) 

0 

Ibb 

tn 

VOJ  i 

M2 

2* 

MOC1  ftni 
n^Di,ou  1, 

0 

*72 

(0 

032) 

n  7nft 
U .  /UO 

(0 

055) 

0 

209 

t  n 

M2 

27 

uoe 1 oni 

0 

589 

(r 

032) 

-1  QA^ 

(0 

11*) 

0 

oi  1 
Zll 

(n 

M2 

25 

Moeooni 

0 

517 

(0 

035) 

"1 . 239 

(0 

089) 

0 

000 

tn 

V  ClO  } 

M2 

17 

MOftoAni 

0 

920 

(0 

0*0) 

"1 . 326 

(0 

068) 

n 

u 

occ 

(0 

025) 

Ml 

17 

0 

878 

(0 

060) 

0  .217 

(0 

0*3) 

0 

*77 

(0 

013) 

M2 

35 

MOROCnO 

0 

598 

(0 

0*5) 

1 .756 

(0 

1*1) 

0 

365 

(0 

010 ) 

M2 

36 

N2S26C1 

0 

756 

(0 

039) 

0 .  *32 

(0 

038) 

0 

233 

(0 

012) 

Ml 

38 

0 

t>6* 

(0 

027) 

0 .707 

fO 

035) 

0 

000 

(n 

UUU  ) 

Ml 

*3 

Moeoini 
n^D«>iu  i. 

0 

75* 

(0 

032) 

(0 

033) 

0 

000 

/  n 

nnn  ^ 
UUU  ^ 

M2 

37 

Moeooni 
A^D«>^U  i. 

0 

973 

(0 

050) 

1  .  OH  0 

(0 

080) 

0 

361 

k  u 

026) 

M2 

18 

n^b J^U^ 

0 

659 

(0 

0*2) 

-n  k'XL 

\i  .HOH 

(0 

0*1) 

0 

352 

0  ID  ^ 

M2 

19 

0 

BOO 

(0 

028) 

n  Qfifi 

U  .  uOO 

(0 

0*0) 

0 

000 

tn 

nnn  ^ 

UUU  / 

M1 

ni 

*7 

1 

578 

<o 

082) 

U .  Moo 

(0 

038) 

0 

216 

0  XJ  ) 

39 

Mocccn 1 

0 

775 

(0 

060) 

U .  J^b 

(0 

0*1) 

0 

2** 

(0 

017) 

no 

01 

0 

917 

(0 

039) 

1 .65* 

(0 

08.S) 

0 

102 

(0 

.009) 

M2 

*8 

N2C8001 

1 

691 

(0 

062) 

0.639 

(0 

054) 

0 

18* 

(0 

012) 

M2 

*7 

NZeQOOl 

0 

938 

(0.0*6) 

-0.398 

(0 

03*) 

0 

169 

(0 

016) 

M2 

22 

N270301 

0 

9*2 

(0 

.036) 

-l.*03 

(0 

063) 

0 

1*0 

(0 

.026) 

Ml 

30 

N270302 

1 

586 

(0 

059) 

0.119 

(0 

031) 

0 

067 

(0 

009) 

Ml 

31 

N271301 

1 

37* 

(0 

120) 

0  185 

(0 

0*8) 

0 

261 

(0 

01*) 

M3 

32 

N27SS01 

1 

030 

(0 

.033) 

-0.759 

(0 

035) 

0 

OOC 

(0 

000) 

Ml 

23 

N278502 

0 

895 

(0 

.032) 

-0.559 

(0 

030) 

0 

000 

(0 

000) 

Ml 

2* 

N278503 

0 

900 

(0 

030) 

-0.831 

(0 

037) 

0 

000 

(0 

.000) 

Ml 

25 

H278001 

1 

129 

(0 

.056) 

-0.229 

(0 

.03*) 

0,232 

(0 

.015) 

M2 

23 

N278002 

1 

162 

(0 

.065) 

0.01* 

(0 

.036) 

0 

236 

(0 

016) 

M2 

*2 

H278003 

1 

.921 

(0 

092) 

0.365 

(0 

051) 

0 

227 

(0 

.013) 

M2 

** 

II278905 

1 

178 

(0 

.0*6) 

1.053 

(0 

.063) 

0 

.283 

(0 

.010) 

Ml 

** 

N280401 

0 

550 

(0 

.026) 

-1  313 

(0 

.067) 

0 

000 

(0 

.000) 

M2 

30 

N281401 

0 

685 

(0 

032) 

-0.2*j 

(0 

027) 

0 

.109 

(0 

.015) 

M2 

29 

H282801 

1 

.806 

(0 

.05*) 

1.31G 

(0 

.075) 

0 

206 

(0 

010) 

Ml 

*8 

N286001 

0.766 

(0.035) 

-0.9** 

/  *^ 
'  w 

.051) 

0 

.169 

(0 

OZO) 

Ml 

19 

671 


705 


Table  F-10  (continued) 


1988  IRT  Parameters,  Mathematics  Trend,  Age  17 


NAEP  ID 

t 

S.E. 

H286002 

0.8SS 

(0.032) 

N2e6301 

1.3S0 

(0.071) 

N286302 

1.088 

(0.0S6) 

R285501 

1.142 

(0.049) 

lf286S02 

1.797 

(0.097) 

R287101 

1.3S8 

(0.060) 

N287102 

I.IU 

(O.OSO) 

N287301 

0.793 

(0.030) 

N287302 

0.994 

(0.031) 

B  S.E.  C 

-1.658  (0.071)  0.121 

-0.450  (O.043)  0.221 

-0.439  (0.044)  0.289 

-0.847  (0.049)  0.149 

-0.123  (0.038)  0.191 

-0.382  (0.037)  0.202 

-0.556  (0.040)  0.172 

0.120  (0.022)  0.000 

1.226  (0.048)  0.000 


AGE 

17 

S.E. 

BLOGC 

ITEM 

(0.027) 

Ml 

20 

(0.017) 

M2 

33 

(0.018) 

Ml 

22 

(0.021) 

M2 

34 

(0.013) 

Ml 

34 

(0.014) 

Ml 

29 

(0.018) 

M2 

32 

(C.OOO) 

Ml 

45 

(0.000) 

Ml 

46 

2 

7nB 


Table  F-11 


1988  IRT  Parameters,  Science  Trend,  Age  9 

AGE  9 


A 

a, 

0 

b  .  fc  , 

c 

p .  fc , 

BLCXTK 

ITEM 

Hinnnni 

11^  UU  UU  X 

(1  RSn 

/n  ASR^ 
\  U  .  \JJO  ) 

/A  1 n  Q \ 

0 

237 

(0 . 030 ) 

SI 

6 

\U. UD9/ 

2 . 732 

(0 . 643 ) 

0.460 

(3.016) 

SI 

IS 

(1 

1 .  9U9 

/ A   10Q \ 

0 

424 

(0 . 015) 

SI 

16 

*0 . 130 

(0.055) 

0 

340 

(0.021) 

SI 

8 

1  2A6 

/A   AQ9 \ 

..1    0  1  A 

/A    1 1 7 \ 

0 

417 

(0 . 035 ) 

SI 

9 

1  A9P 

/  n  1 ^ 

U .  /  J  J 

/ A  nao \ 
VU. U09 ) 

0 

280 

(0 . 027 ) 

SI 

10 

U  •  JoO 

/n  no^ 

OA  1 
1  .  9*»  1 

1  n  oo**  \ 

0 

422 

(0 . 036 ) 

SI 

11 

1  •  XOH 

/n  ^Ofi^ 

\  U  .  IF9O  ^ 

*U . oOx 

1 n  n7  a \ 

0 

322 

(0 . 026 ) 

SI 

12 

V  U*  U9 J  ^ 

«.  A    7  AO 

/ A  non \ 

0 

390 

(0. 027) 

SI 

13 

n^uu Jul 

n 

U .  OVo 

1  n  nai \ 

0 

330 

(0.018) 

SI 

14 

n  AAA 

_  n  ono 
U .  ZUZ 

/  n  nA  A  \ 

0 

225 

(0 . 021 ) 

SI 

17 

11^  W  /v  X 

A  741 

^  U • UDD  ^ 

U .  U/U 

/ n  nA A \ 

0 

202 

(0.019) 

SI 

18 

\V  t  U^9  ^ 

1  fin  A 

tn  ORfi^ 

0 

253 

(0.015) 

SI 

19 

^  U  .  U  J  J  ^ 

n  70Q 

U  .  /Z9 

/  n  nao  \ 

0 

210 

(0. 016) 

SI 

20 

H&D11D1 

Ii"t  W  X  XV  X 

0 . 292 

(0 . C48 } 

1  7'*7 

Xm  f  Of 

tn  OAA^ 

0 

275 

/ n  ni c  ^ 
. UlD ; 

21 

N&ni  9(11 

11^  w  X^V  X 

A  AS1 

Xf  m  OJX 

tn  nAn^ 

/ n  0%  ^  ^ 
V  u .  zu  ^ 

0 

243 

(0.011) 

SI 

22 

0 . 50A 

(0 . 060 ) 

1 . 478 

/  n  1  fi*)  ^ 
V  u . 100  J 

0 

269 

/  n  n  1  A  \ 
. U Is  ^ 

01 

o** 
ZJ 

Nini ^ni 

/n  n47^ 

n   OA  Q 

/ n  nc A  ^ 

0 

347 

/  n  A 1  0  \ 
(U . Ul9 ; 

S2 

1 

0. 599 

(0 . 058 ) 

-  i  AGO 

t(\  1  ^(1 1 

V  IF  .  UIF  ^ 

0 

207 

/  n  Ati ^ 
\\J ,  voo ) 

S2 

z 

tn  n<^Qi 

I  .  33d 

/  n  1 1  0  \ 
VU . llo ) 

0 

452 

(0 . 0 18 ) 

S2 

4 

0 . 299 

tn  nsQi 

1  n*)  ^ 

tn  ono ^ 

0 

443 

/  n  n  1  7  \ 
. U 1/ ; 

S2 

5 

N&(11  A(11 

(1  RAR 
U  •  DOD 

tn  lAQi 

^  U  *  XW  f 

-n  n*) ^ 

/ n  n^7  ^ 

0 

447 

/ n  no 1 \ 
. UZl ; 

S2 

6 

N4(11  A(19 

n  S7n 

tn  nA9> 

-n  ORO 

/  n  1 A  7  ^ 
(U . Is / j 

0 

432 

/ n  nofi \ 

S2 

7 

N4(11  An*) 

(1  4SS 

tn  n7s^ 

■  n  07Q 

/n  nRfi^ 

^  IF .  UDO  J 

0 

440 

/ n  noo  \ 

S2 

8 

0 .  3A6 

(0 . 068 ) 

1  RQA 
1 .  D9O 

/n  '%'%A^ 
^  u  .  000 } 

0 

424 

tn  n^^i\ 

\\J  ml)  ID  ) 

S2 

f% 
9 

H401 dOI 

0 . 469 

(0 . 072 ) 

1  A^^ 

tn  0Q1 ^ 

\  U  . ^9l J 

0 

318 

/  n  n  1  c  ^ 

CO 

1  n 
lU 

H409001 
v^vw  X 

0 . 935 

(0 . 091 ) 

-1    (14  S 

/n   1 1  A  ^ 
\v  1 110  J 

0 

381 

/A  n'*o\ 

CO 

11 

1 . 22A 

(0 . 106 ) 

/(I   1  1  S  ^ 
\\f  t  XX  J  } 

0 

386 

tn  nt A ^ 

CO 

1  0 

IZ 

N40200S 

0 , 712 

(0. 103 ) 

"0 . 310 

(0 . 091 ) 

0 

411 

tn  (IORI 

\v  t  VF^D  J 

CO 

oZ 

i  C 

13 

H402101 

41  ^  V     X  V  X 

0 .562 

(0 , 061 ) 

U  .  00£t 

/A  nsi 1 

\M  t vJX / 

0 

206 

tn  ti*>o\ 

CO 

oZ 

ID 

H432201 

C.lc^31 

(0.039) 

u .  joo 

(0 . 067 ) 

0 

2^5 

tn  Ai 0 ^ 

CO 
oZ 

1  7 
1  / 

H402401 

0.253 

(0.051) 

2 . 764 

(0 . 561 ) 

0 

235 

tn  (11^1 

V  U  .  IF13  J 

CO 
oZ 

1  fi 

H402S01 

&1  "ff  V     «^  V  X 

0 . 62?m 

(0 , 090 ) 

0  RQO 

(0.407) 

0.258 

tn  (11 1  ^ 

^  U  .  VFl  1 J 

CO 
oZ 

1  Q 

19 

H402602 

0.401 

(0 .063) 

~0 . 686 

(0.117) 

0. 

439 

tn  nooi 

CO 
oZ 

0 1 
Zl 

H402701 

41  "ff  V  ^  /  V  X 

0 .453 

(0.058) 

1. 980 

(0.261) 

0 

199 

(0 . 0 13 ) 

^0 

£tO 

N402801 

1 .084 

(0.083) 

2. 031 

(0. 189) 

0. 

161 

tn  nnoi 

<!•> 
0^ 

0-4 

N402901 

0.373 

(0.094) 

4 . 734 

(1. 194 ) 

0. 

185 

(0 . 0 10 ) 

^0 

OS 

NA03001 

0.422 

(0. 062) 

■5. 043 

(0 . 745) 

0. 

238 

(0 . 053 ) 

<?*) 

1  0 

X£t 

NA03101 

0. 638 

(0.062) 

■3. 422 

(0.342) 

0 

232 

(0.051) 

<?*) 

13 

NA03201 

0.404 

(0.048) 

-3.042 

(0.368) 

0 

212 

(0.039) 

S3 

14 

NA03202 

0.291 

(0.038) 

-1.195 

(0.161) 

0.238 

(0.024) 

S3 

15 

NA03301 

0.624 

(0.056) 

-1.079 

(0.105) 

0. 

218 

(0.029) 

S3 

16 

NA0340I 

0.355 

(0.043) 

0.596 

(0.089) 

0 

338 

(0.017) 

S3 

17 

N403S01 

0.563 

(0  067) 

0.257 

(0.057) 

0 

400 

(0.019) 

f?3 

18 

NA03S02 

0.551 

(0.059) 

-1.918 

(0.211) 

0 

(0.034) 

S3 

19 

N403S03 

0.412 

(0.060) 

0.152 

(0.054) 

0 

;09 

(0.020) 

S3 

20 

NA03601 

0.811 

(0.069) 

0.534 

10.065) 

u 

254 

(0.016) 

S3 

21 

N403701 

3.290 

(0.390) 

-0.287 

(0.108) 

0. 

312 

(0.021) 

S3 

22 

N403702 

3.150 

(0.247) 

-0.496 

(0.118) 

0 

374 

(0.023) 

S3 

23 

N403703 

2.076 

(0.204) 

-0.326 

(0.077) 

0 

302 

(0.021) 

S3 

24 

NA03801 

0.359 

(0.057) 

1.082 

(0.180) 

0 

428 

(0.017) 

S3 

25 

NA03803 

0.497 

(0.056) 

-0.991 

(0.119) 

0. 

393 

(0.026) 

S3 

27 

NA03804 

0.484 

(0.063) 

-0.506 

(0.080) 

0. 

408 

(0.023) 

S3 

28 

NA03901 

0.653 

(0.056) 

-0.309 

(0.046) 

0 

193 

(0.023) 

S3 

29 

NA04001 

0.203 

(0.036) 

1.764 

(0.317) 

0. 

223 

(0.016) 

S3 

30 

HA0A201 

0.425 

(0.050) 

1.363 

(0.165) 

0. 

216 

(0.015) 

S3 

31 

673 

7.07 


Table  F-12 

1988  IRT  P?-'ameters,  Science  Trend,  Age  13 


HAEP  ID 

R400201 
N401201 
N404501 
N404601 
H404701 
K404702 
N404801 
N404802 
N404603 
N404901 
N40S001 
H40S101 
N40S201 
N405301 
H40S401 
N40S301 
H40S601 
N40S701 
N405801 
:i405901 
H406001 
H406101 
N406201 
H406301 
N406302 
H406303 
N406304 
N406401 
H406402 
N4064d3 
N406404 
lt406405 
N406501 
N406601 
H406701 
N406801 
H405802 
H405803 
H406804 
H40680S 
H406806 
H406901 
N407001 
N407101 
H407201 
H407301 
H407302 
H407601 
H407701 
H407d01 
N407901 
R408001 
H408201 
R406301 
N408302 
N408303 
H408304 
H408401 
H408501 
N408S02 
H408601 
N408701 
11408801 
H408901 


0.464 
0.5^4 
I  153 
0.318 
0.601 
0.449 
1.372 
1.610 
0.956 
0.691 
0.349 
0.764 
0.bl5 
0.623 
0.801 
0.628 
0.233 
1.012 
0.493 
0.637 
0.455 
0.531 
0.360 
0.356 
0.386 
0.6C6 
0.471 
0.504 
0.861 
0.753 
0.910 
0.628 
0.495 
0.491 
0.576 
1.128 
0.342 

0.  816 

1.  C57 
1.037 
0.440 
0.613 
0.263 
0.817 
0.470 
0.319 
0.346 
0.433 
0.564 
0.666 
0.383 
0.848 
0.567 
0.788 
0.708 
0.647 
0.971 
0.240 
0.733 
0.390 
0.388 
0.346 
0.174 
0.743 


S.E. 

0.041) 
0.049) 
0.055) 
0.039; 
0.043) 
0.041) 
0.085) 
0.140) 
0.078) 
0.051) 
0.038) 
0.052) 
0.036) 
0.049) 
0.053) 
0.052) 
0.034) 
0.067) 
0.044) 
0.049) 
0.107) 
0.120) 
0.089) 
0.052) 
0.051) 
C.063) 
0.066) 
0.066) 
0.090) 
0.074) 
0.111) 
0.066) 
0.043) 
0.044) 
0.049) 
0.074) 
n.047) 
0.074) 
0.073) 
0.097) 
0.053) 
0.052) 
0.035) 
0.055) 
0.041) 
0.039) 
0.046) 
0.044) 
0.044) 
0.055) 
0.037) 
0.050) 
0.070) 
0.061) 
0.065) 
0.060) 
0.079) 
0.032) 
0.056) 
0.040) 
0.035) 
0.038) 
0.030) 
0.079) 


B 

-1.666 
0.415 
-2.021 
-0.641 
-1.538 
-0.140 
-1.624 
-0.514 
0.240 
-0.629 
0.200 
0.968 
-0.124 
1.251 
1.138 
-0.031 
1.041 
0.715 
1.324 
1.658 
4.846 
4.384 
5.620 
-1.563 
-0.408 
1.470 
1.354 
-0.157 
0.303 
-1.328 
-0.305 
-0.528 
O.bi? 
-0.855 
0.093 
-1.417 
0.687 
-0.660 
-1.014 
1.523 
0.226 
0.019 
0.158 
2.218 
0.437 
1.672 
1.817 
1.743 
1.273 
2.158 
0.849 
1.268 
3.245 
0.97U 
-1.545 
-1.697 
-1.384 
-1.476 
-0.896 
1.337 
-1.071 
-0.101 
0.655 
0.274 


S.E. 

(0.151) 
(0.051) 
(0.119) 
(0.084) 
(0.117) 
(0.033) 
(0.136) 
(0.077) 
(0.049) 
(0.057) 
(0.037) 
(0.077) 
(0.033) 
(0.107) 
(0.087) 
(0.035) 
(0.153) 
(0.065) 
(0.124) 
(0.137) 
(1.148) 
(1.008) 
(1.399) 
(0.231) 
(0.069) 
(0.166) 
(0.200) 
(0.050) 
(0.062) 
(0.142) 
(0.067) 
(0.075) 
(0.064) 
(0.082) 
(0.034) 
(0.114) 
(0.104) 
(0.074) 
(0.086) 
(0.181) 
(0.050) 
(0.034) 
vO.038) 
(0.168) 
(0.050) 
(0.208) 
(0.245) 
(0.173) 
(0.107) 
(0.189) 
(0.089) 
(0.08'/) 
(0.415) 
(0.089) 
(0.152) 
(0.163) 
(0.129) 
(0.199) 
(0.077) 
(0.140) 
(0.102) 
(0.031) 
(0.117) 
(0.0J5) 


0.206 
0.249 
0.164 
0.228 
0.194 
0.201 
0.422 
0.360 
0.321 
0.209 
0.'ai4 
0.)99 
0.182 
0.199 
0.18\ 
0.197 
0.198 
0.185 
0.166 
0.158 
0.174 
0.207 
0.099 
0.430 
0.428 
0.392 
3.419 
.461 
.405 
.419 
.457 
.402 
0.170 
0.175 
0.240 
0.396 
.445 
.382 
.371 
.550 
.423 
.231 
.182 
.126 
0.207 
0.234 
0.270 
0.180 
0.144 
0.199 
0.168 
0.176 
0.206 
0.298 
0.408 
0.415 
0.414 
0.223 
0.205 
0.154 
0.153 
0.212 
0.234 
0.445 


(0.029) 
(0.016) 
(0.042) 
(0.021) 
(0.029) 
{0.018) 
(0.043) 
(0.022) 
(0.016) 
(0.022) 
(0.017) 
(0.012) 
(0.019) 
(0.012) 
(0.011) 
(0.019) 
(0.016) 
(0.013) 
(0  012) 
(0.011) 
(0.008) 
(0.008) 
(0.007) 
(0.026) 
(0.021) 
(0.013) 
(0.015) 
(0.020) 
(0.018) 
(0.031) 
(0.022) 
(0.025) 
(0.016) 
(0.023) 
(0.016) 
(0.036) 
(0.015) 
(0.022) 
(0.027) 
(0.011) 
(0.017) 
(0.017) 
(0.019) 
(0.009) 
(0.015) 
(0.013) 
(0.014) 
(0.012) 
(0.012) 
(0.010) 
(0.015) 
(0.010) 
(0.009) 
(0  013) 
(0.033) 
(0.031) 
(0.034) 
(0.0P.2) 
(0.025) 
(0.0x3) 
(0.022) 
(0.018) 
(0.017) 
(0.015) 


AGE  13 
BLOCK  ITEM 


SI 

16 

SI 

28 

SI 

12 

SI 

13 

SI 

14 

SI 

15 

SI 

20 

SI 

21 

SI 

22 

SI 

17 

SI 

23 

SI 

24 

SI 

25 

SI 

26 

SI 

27 

SI 

29 

SI 

30 

SI 

31 

SI 

32 

SI 

33 

SI 

34 

SI 

35 

SI 

36 

S2 

10 

S2 

11 

S2 

12 

S2 

13 

S2 

14 

S2 

15 

S2 

16 

S2 

17 

S2 

18 

S2 

19 

S2 

20 

S2 

21 

S2 

22 

S2 

23 

S2 

24 

S2 

25 

S2 

26 

S2 

27 

S2 

28 

S2 

29 

S2 

30 

S2 

31 

S2 

32 

S2 

33 

S2 

35 

S2 

37 

S2 

38 

S2 

39 

S2 

34 

S2 

40 

S3 

10 

S3 

11 

S3 

12 

S3 

13 

S3 

14 

S3 

15 

S3 

16 

S3 

17 

S3 

18 

S3 

19 

S3 

20 

ERIC 


674 


7n8 


Table  F-12  (continued) 


1988  IRT  Parameters,  Science  Trend,  Age  13 

AGE  13 


NAEP  ID 

A 

1^ 

I 

S.E. 

c 

S.E. 

BLOCK 

ITEM 

N408g02 

0. 

eee 

(0.069) 

-1 

740 

(0.149) 

0 

410 

(0.038) 

S3 

21 

N^0e903 

0. 

656 

(0.066) 

0 

434 

(0.062) 

0 

404 

(0.015) 

S3 

22 

NA0e904 

0. 

540 

(0.060) 

0 

877 

(0.107) 

0 

411 

(0.014) 

S3 

23 

N409001 

0 

599 

(0.045) 

-0 

364 

(0.040) 

0 

163 

(0.018) 

S3 

24 

N409101 

0 

635 

(0.045) 

-1 

494 

(0.113) 

0 

239 

(0.029) 

S3 

25 

N409102 

0 

556 

(0.047) 

0 

178 

(0.036) 

0 

229 

(0.016) 

S3 

26 

N409103 

0 

518 

(0.059) 

2 

017 

(0.235) 

0 

306 

(0.011) 

S3 

27 

N409201 

0 

292 

(0.039) 

0 

393 

(0.061) 

0 

261 

(0.017) 

S3 

28 

N409301 

0 

706 

(0.056) 

-0 

145 

(0.035) 

0 

165 

(0.918) 

S3 

29 

N409501 

0 

607 

(0.052) 

2 

148 

(0.191) 

0 

134 

(0.009) 

S3 

33 

N409601 

0 

706 

(0.061) 

1 

717 

(0.162) 

0 

290 

(0.011) 

S3 

34 

N409701 

0 

633 

(0.060) 

2 

485 

(0.248) 

0 

165 

(0.009) 

S3 

35 

ERIC 


Talile  F-13 


1988  IRT  Parameters,  Science  Trend,  Age  17 


KAEP  ID 

R400201 
N401201 
K404601 
K405001 
K40S101 

0.5201 
H40S401 
H405501 
N406001 
H406101 
N406201 
K406301 
H406302 
K406303 
K40630v 
K406401 
N406402 
H^06403 
K406404 
K406405 
K406601 
H406801 
K406802 
K;o6803 
K406004 
K406805 
K406806 
K406901 
N407001 
K407101 
K407201 
IU07301 
R407302 
N407401 
K407403 
N407404 
N407701 
K408101 
K408301 
K408302 
N408303 
K408304 
H408601 
H408801 
H408001 
K408902 
K408003 
K408004 
H400301 

09501 
K400001 
H410003 
N410004 
N410101 
KA10102 

10103 
H410201 
N410401 

10501 
K410601 

10602 
K4l06g3 

10604 
N4107U1 


0.543 
0.613 
0.542 
0.462 
0.595 
0.444 
0.619 
0.584 
0.471 
0.494 
0.658 
0.334 
0.420 
0.506 
0.511 
0.632 
0.67S 
0.815 
0.721 
0.637 
0.547 
0.672 
0.452 
0.575 
0.709 
0.458 
0.3S6 
0.500 
0.333 
0.61<S 
0.500 
0.346 
0.445 
0.652 
0.581 
0.714 
0.450 
0.625 
0.834 
0.457 
0.543 
0.640 
0.426 
0.505 
0.769 
0.836 
0.563 
0.586 
0.625 
0.714 
0.867 
0.509 
0.499 
0.626 
0.433 
0.566 
0.491 
0.396 
0.415 
1.05/ 
0.430 
0.768 
0.414 
0.542 


0.116) 
0.124) 
0.118) 
0.098) 
0.111) 
0.095) 
0.104) 
0.121) 
0.115) 
0.142) 
0.130) 
0.091) 
0.105) 
0.129) 
0.136) 
0.156) 
0.189) 
0.189) 
0.1b2) 
0.171) 
0.115) 
0.157) 
0.121) 
0.136) 
0.147) 
0.117) 
0.105) 
0.109) 
0.079) 
0.126) 
0.106) 
0.083) 
0.110) 
0.143) 
0.151) 
0.166) 
0.098) 
0.124) 
0.186) 
0.119) 
0.122) 
0.162) 
0.091) 
C.lOl) 
0.168) 
0.165) 
0.127) 
0.135) 
0.122) 
0.129) 
0.168) 
0.121) 
0.129) 
0.158) 
0.113) 
0.139) 
0.119) 
0.093) 
0.086) 
0.208) 
0.122) 
0.170) 
0.110) 
0.120) 


B 

-1.669 
-0.226 
-0.565 
-0.305 
0.272 
-0.703 
0.b31 
-0.295 
2.129 
2.885 
2.184 
-1.322 
-0.246 
0.383 
-0.276 
-0.678 
-0.075 
-1.522 
-1.204 
-0.963 
-0,915 
-1.921 
1.261 
-1.248 
-1.539 
0.473 
0.270 
-0.532 
-0.920 
1.878 
0.120 
0.510 
0.917 
0.348 
-0.258 
-1.370 
0.883 
1.626 
-0.241 
-1.685 
-2.012 
-1.^85 
-1.329 
-0.3i\0 
-1.203 
-1.022 
-0.172 
-0.374 
-1.324 
1.100 
-0.931 
-1.988 
-1.225 
-0.700 
-0.401 
-1.408 
1.690 
0.066 
-0.420 
2.077 
-2.476 
1.333 
-2.139 
0.633 


S.E. 

(0.370) 
(0.097) 
(0.150) 
(0.102) 
(0.103) 
(0.168) 
(0.138) 
(0.108) 
(0.536) 
(0.854) 
(0.457) 
(0.371) 
10.118) 
(0.147) 
(0.133) 
(0.205) 
(0.124) 
(0.386) 
(0.333) 
(0.292) 
(0.210) 
(0.471) 
(0.367) 
(0.314) 
(0.344) 
(0.164) 
(0.128) 
(0.142) 
(0.222) 
(0.410) 
(0.084) 
(0.147) 
(0.249) 
(0.136) 
(0.137) 
(0.349) 
(0.214) 
(0.344) 
(0.125) 
(0.456) 
(0.470) 
(0.426) 
(0.295) 
(0.105) 
(0.292) 
(0.422) 
(0.112) 
(0.135) 
(0.274) 
(0.225) 
(0.209) 
(0.486) 
(0.334) 
(0.207) 
(0.144) 
(0.365) 
(0.476) 
(0.088) 
(0.118) 
(0.507) 
(0.714) 
(0.336) 
(0.577) 
(0.209) 


0.196 
0.229 
0.197 
0.196 
0.235 
0.152 
0.145 
0.196 
0.197 
0.214 
0.116 
0.410 
0.401 
0.397 
0.395 
0.395 
0.391 
0.395 
0.393 
0.397 
0.151 
0.396 
0.403 
0.396 
0.391 
0.408 
0.406 
0.196 
0.155 
0.150 
G.153 
0.204 
0.246 
0.375 
0.393 
0.395 
0.152 
0.142 
0.381 
0.401 
0.398 
0.396 
0.164 
0.198 
0.393 
0.395 
0.392 
0.396 
0.149 
0.133 
0.191 
0.400 
0.401 
0.394 
0.404 
0.390 
0.1^5 
0.244 
0.150 
0.229 
0.405 
0.338 
0.405 
0.201 


0.049) 
0.040) 
0.043) 
0.039) 
0.035) 
0.040) 
0.031) 
0.041) 
0.026) 
0.02S) 
0.022) 
0.047) 
0.04Z> 
0,037) 
0.044) 
O.049) 
0.043) 
C.OStI) 
Q  056) 
0.054) 
0  0A2) 
^.059) 
).033) 
J. 051) 
0.056) 
0.037) 
0.040) 
0.042) 
0.042) 
0.026) 
0.035) 
0.036) 
0.03S) 
0.036) 
0.046) 
0.057) 
0.032) 
0.026) 
0.041) 
0.056) 
0.057) 
0.058) 
0.043) 
0.03&) 
0.053) 
0.062) 
0.041) 
0.043) 
0.044) 
0.028) 
0.046) 
0.055) 
0.050) 
0.046) 
0.043) 
0.053) 
0.030) 
0.039) 
0.039) 
0.025) 
0.058) 
0.029) 
0.055) 
0.033) 


AGE  17 
BLOCK  ITEM 


SI 
SI 
SI 
SI 
S3 
SI 
S3 
S3 
SI 
SI 
SI 
51 
SI 
SI 

o2 
S2 
S2 
S2 
S2 
SI 
S2 
S2 
S2 
S2 
S2 
S2 
S2 
S2 
S2 
S2 
S2 
S2 
S2 
S2 
S2 
S2 
SI 
S3 
S3 
S3 
S3 
SI 
S3 
S3 
S3 
S3 
S3 
31 
SI 
SI 
SI 
SI 
SI 
SI 
SI 
SI 
S2 
S2 
S2 
S2 
S2 
S2 
S2 


12 
30 
13 
29 
14 
31 
19 
21 
33 
35 
37 
21 
22 
23 
24 
10 
11 
12 
13 
14 
28 
16 
17 
18 
19 
20 
21 
27 
33 
38 
32 
36 
37 
28 
30 
31 
35 
38 
10 
11 
12 
13 
19 
24 
15 
16 
17 
18 
20 
34 
18 
16 
17 
25 
26 
27 
32 
15 
22 
23 
24 
25 
26 
34 
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Table  F-13  (continued) 


1988  IRT  Parameters,  Science  Trend,  Age  17 


S.E. 

H41O801 

O.SA«^ 

(0.124) 

H410901 

0.707 

(0.134) 

N411001 

0,5*5 

(0.145) 

N411101 

0.508 

(0.096) 

11201 

0.566 

(0.105) 

n;ii3oi 

0.469 

(0.139) 

K^ll^Ol 

0.846 

(0.151) 

K^llSOl 

0.860 

(0.125) 

H411S02 

0.619 

(0.131) 

11801 

0.609 

(0.108) 

N411701 

0.745 

(0.119) 

11801 

1.069 

(0.175) 

K411901 

0.76/ 

(0.122) 

N^lZOOl 

0.572 

(0.119) 

a  c 

1.554  fO.376)  0.193 

1.777  (0.367)  0.155 

2.730  (0.751)  0.193 

0.255  (0.093)  0.150 

0.490  (0.126)  0.195 

3.814  (1.163)  0.120 

0.534  (0.137)  0.152 

1.749  (0.300)  0.179 

-1.037  (0.240)  0.237 

1.227  (0.244)  0.184 

1.395  (0.256)  0.169 

0.650  (0.161)  0.167 

1.429  (0.261)  0.142 

2.048  (0.453)  0.187 


AGE 

17 

S.E. 

BLOCK 

ITEM 

(0.030) 

S2 

39 

(0.024) 

ZZ 

40 

(0.024) 

o2 

41 

(0.035) 

S3 

22 

(0.033) 

S3 

23 

(0.023) 

S3 

20 

(0.030) 

S3 

25 

(0.024) 

S3 

26 

(0.048) 

S3 

27 

(0.030) 

S3 

28 

(0.027) 

S3 

29 

(0.031) 

S3 

30 

(0.025) 

S3 

31 

(0.029) 

S3 

32 
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Appendix  G 


NAEP  REPORTING  SUBGROUPS 


DSEX  (Gender) 

The  variable  SEX  on  the  student  file  is  the  gender  of  the  student  being 
assessed,  as  taker,  from  school  records.    Fol  a  few  studencs,  data  for  this 
variable  was  massing  and  was  imputed  by  ETS  after  the  assessment.  The 
resulting  variable  DSEX  on  the  student  file  contains  a  value  for  every  student 
and  is  used  for  gender  comparisons  among  students. 


RACE    (Observed  Race/Ethnicity) 

The  variable  RACE  on  the  student  file  is  the  race/etanicity  of  the 
student  being  assessed,  as  observed  and  recorded  by  the  exercise 
cdminisc.  ator .    Observed  race/ethnicity  was  used  in  NAEP  assessments  befoLe 
1984.     This  variable  is  used  for  race/ethnicity  subgroup  comparisons  to 
assessments  conducted  before  1^84. 


DRACE  (Imputed  Race/Ethnicity) 

The  variable  DRACE  on  the  student  file  is  an  imputed  definition  of 
race/ethnicity,  derived  from  up  to  three  sources  of  information.  This 
variable  is  used  for  race/ethnicity  subgroup  comparisons  within  the  1988 
assessment  and  among  the  1988,  1986,  and  1984^  assessments. 

Two  common  background  items  were  used  in  the  determination  of 
race/ethnicity: 

Conunon  Background  Item  Number  2: 

2.     If  you  are  Hispanic,  what  is  your  Hi  -.panic  background? 
CD      I  am  not  Hispanic. 

CD     Mexican,  Mexican  American,  or  Chicano 
CD      Puerto  Rican 
CD  Cuban 

CD     Other  Spanish  or  Hispanic  background 


^T.te  1984  variable  ETHNIC  contains  the  same  race/ethnicity  categories  as 
the  1988  variable  DRACE.  but  the  imputation  procedure  used  was  slightly 
different. 
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Students  who  responded  to  item  number  2  by  filling  in  the  second,  tnird, 
fourth,  or  fifth  oval  were  considered  Hispanic.    For  students  whc  filled  in 
the  first  oval,  did  not  respond  to  the  item,  or  pic   tded  information  that  was 
illegible  or  could  not  be  classified,  responses  to  item  number  1  were  examined 
in  an  affort  to  determine  race/ethnic  Ity.    Item  nuiaber  1  read  as  follow^^: 


Common  Background  Item  Nixmber  1: 
1.    Which  best  describes  you? 


CD  White 

CD  Black 

CD  Hispanic 

CD  Asian  or  Pacific  Islander 

CD  American  Indian  or  Alaskan  Nati  " 

CD  Other  (What?)   


Students'  race/ethnicity  was  then  assigned  to  correspond  with  their 
selection.     For  students  who  filled  in  the  sixth  oval  ("Other"),  pro^-lded 
illegible  information  or  information  that  could  not  be  classified,  or  did  not 
respond  at  all,  observed  race/ethnicity  (RACE),  if  provided  by  the  exercise 
administrator,  was  used. 

Imputed  race/ethnicity  could  not  be  determined  for  students  who  did  not 
respond  to  background  items  1  or  2  and  for  whom  an  observed  ruce/ethnicity  was 
not  provided. 


STOC    (Size  and  type  of  community) 

NAEP  assigned  ea'^h  participating  school  to  one  of  seven  size  and  type  of 
community  categories  designed  to  provide  information  about  the  communities  in 
which  the  schools  are  located. 

The  STOC  reporting  categories  consist  of  three  "extreme"  types  of 
communities  and  four  "residual"  community  sizes.     Schools  were  placed  Into 
STOC  categories  on  ♦rhe  basis  of  information  about  the  type  of  community,  the 
size  of  its  population  (as  of  the  1980  Census),  and  an  occupational  profi.le  of 
residents  provided  by  school  principals  before  the  assessment.     The  principals 
completed  estimates   >f  the  percentage  of  students  whose  parents  fit  into  ea^h 
of  six  occupational  -ategories. 

Two  versions  of  STOC  were  created:    a  weighter*  version  usec^  for  cross- 
sectional  analyses  and  an  unweigh*"*^  version,  whi  :h  was  the  version  used 
historically.    For  both  versions,  sC    ^Is  were  ranked  in  order  based  on 
principals'  responses  about  the  type  of  cumraunity,  size  of  its  jopulaclop.  and 
occupational  profile  of  the  students'  paren^s.     For  the  weighted  version  of 
STOC,  schools  were  assigned  to  the  extreme  STOC  categories  (1,  2,  and  3)  so  as 


^In  the  bridge  to  1984,  the  race/ethnicity  response  choices  for  this  item 
were  in  a  different  order. 
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to  give  the  result  that  10  percent  of  sampled  students  (weighted)  were 
enrolled  in  schools  in  each  such  category.    For  the  unweighted  version,  the 
extreme  10  percent  of  the  schools  (unweighted)  v,ere  assigned  to  each  of  the 
extreme  STOC  categories.     For  both  versions  of  STOC,  the  remaining  schools 
were  classified  according  to  one  of  the  four  residual  STOC  categories.  The 
extreme  STOC  categories  are  as  follows: 

STOC  1  -  Extreme  Rural:    This  category  was  used  for  schools  in 
rural  areas  where  the  principal  reported  that  a  high  proportion  of 
students'  parents  were  farmers  or  farm  workers  and  a  low  proportion  were 
professional,  managerial,  or  factory  workers.    At  least  some  of  the 
students  in  each  of  these  schools  were  from  open  country  or  places  with 
a  population  of  fewer  than  2,500  and  no  students  were  from  a  place  with 
population  in  excess  of  10,000. 

STOC  2  -  Low  Metro:    The  low  metro  STOC  category  was  used  for 
schools  in  areas  where  the  principal  reported  that  a  high  proportion  of 
students'  parents  were  either  not  regularly  employed  or  on  welfare  and  a 
low  proportion  were  employed  in  professional  or  managerial  positions. 
The  schools  in  STOC  2  were  located  in  cities,  or  the  urbanized  area  of 
cities,  with  a  population  greater  than  200,000. 

STOC  3  -  High  Metro:    High  metro  schools  were  located  in  areas 
where  the  principal  reported  that  a  high  proportion  of  students'  parents 
were  employed  in  professional  or  managerial  positions  and  a  low 
proportion  were  factory  or  farm  workers,  not  regularly  employed,  or  on 
welfare.    STOC  3  schools  were  located  in  cities  or  the  urbanized  area  of 
cities  with  populations  greater  than  200,000. 

Schools  that  did  not  fall  into  STOC  1.  2,  or  3  were  classified  according 
to  four  "residual"  STOC  categories,  depending  on  the  size  of  the  community  in 
which  they  were  located.    The  four  residual  STOC  reporting  categories  are  as 
follows : 


STOC  4  -  Main  Big  Citv:     STOC  4  schools  were  located  within  the 

limits  of  cities  with  populations  greater  than  200,000  but  not 
classified  as  high  or  low  metro. 

STOC  5  -  Urban  Fringe:    The  schools  assigned  to  STOC  5  were 

located  in  the  urbanized  area  outside  the  limits  of  cities  with 

^pulations  greater  than  200,000,  but  not  classified  as  low  or  high 
metro. 


STOC  6  -  Medium  Citv:     STOC  6  schools  were  located  in  cities  with 
populations  between  25,000  and  200,000  that  were  not  classified  as 
belonging  to  the  urbanized  area  of  a  big  city. 

STOC  7  -  Small  Place:    The  schools  assigned  to  STOC  7  were  located 
in  communities  with  populations  of  fewer  than  25,000.     These  communities 
were  not  located  in  the  urbanized  areas  of  big  cities  and  were  not 
classified  as  extreme  rural. 
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PARED  (Parental  education) 

The  variable  PARED  on  the  student  file  is  derived  from  responses  to  two 
common  background  items,  B003501  and  5003601.^    Students  were  asked  to 
indicate  the  extent  of  their  father's  education  (itero  B003601)  by  choosing  one 
of  the  following: 

O     He  cid  not  finish  high  school. 

O     He  graduated  from  high  school. 

O     He  had  some  education  after  higV  lool. 

O     He  graduated  from  colli:ge. 

O     I  don't  know. 

Students  were  asked  to  provide  the  same  information  about  the  extent  of 
their  mother's  education  (item  B003501)  by  choosing  one  of  the  following: 

CD      She  did  not  fini.sh  high  school. 

O      She  graduated  from  high  school. 

O      She  had  some  education  after  high  school. 

CD      She  graduated  from  college. 

CD     I  don't  know. 

The  information  was  combined  into  one  parental  education  reporting 
category  as  follows: 

If  a  student  indicated  the  extent  of  education  for  only  one  Darent,  that 
level  was  included  in  the  data.     If  a  student  indicated  the  extent  of 
education  for  both  parents,  the  higher  of  the  two  levels  was  included  in  the 
data.    For  students  who  did  not  know  the  level  of  education  for  both  parents 
or  did  not  know  the  level  of  education  for  one  parent  and  did  not  respond  for 
the  other,  the  parental  education  level  was  classified  as  unknown.     If  the 
student  did  not  respond  for  both  parents,  the  student  was  recorded  as  having 
provided  no  response. 


^In  the  bridge  to  1984,  the  items  used  were  B000601  and  B000701.  Instead 
of  "had  some  education  after  high  school,"  the  third  response  choice  read, 
"went  to  another  school  after  graduating  from  high  school." 

684 


71B 


REGION 


(Region  of  the  <^ountry) 


In  addition  to  overall  responses,  NAEP  computed  data  for  four 
geographical  regions  in  the  United  States.    States  were  assigned  to  i^g,ions  as 
follows: 


Northeast 

Southeast 

Central 

West 

Connecticut 

Alabama 

Illinois 

Alaska 

Delaware 

Arkansas 

Indiana 

Arizona 

District  or 

Florida 

Iowa 

Calitornia 

Columbia 

-.eorgia 

Kansas 

Colorado 

Maine 

Kentucky 

Michigan 

Hawaii 

Mar^/land 

Louisiana 

Minnesota 

Idaho 

Massachusetts 

Mississippi 

Missouri 

Montana 

New  Hampshire 

North  Carolina 

Nebraska 

Nevada 

New  Jersey 

South  Carolina 

North  Dakota 

New  Mexico 

Nev7  York 

Tennessee 

Ohio 

Oklahoma 

Pennsylvania 

Virginia* 

South  Dakota 

Oregon 

Rhode  Island 

West  Virginia 

Wisconsin 

Texas 

Vermont 

Utah 

Virginia* 

Washington 

Wyoming 

*That  part  of  Virginia  that  is  included  in  the  Washington,  DC,  metropolitan 
statistical  area  is  included  '.n  the  Northeast  region;  the  remainder  of  the  state  is 
includtd  in  the  Southeast  region, 

DGRADE,  MODGRD        (Grade  in  school) 

To  enhance  the  usefulness  of  the  data,  in  1984  NAEP  began  sampling 
students  by  grade  as  well  as  by  age.    The  ages  sampled  in  assessments  slice 
1984 — 9,  13,  and  17 — match  the  ages  sampled  in  earlier  assessments.  However, 
some  of  the  modal  grades  (the  grade  attended  by  most  students  o*"  a  particular 
age)  for  the  ages  sampled  have  varied  in  the  last  three  assessments  because  of 
changes  in  how  student  age  was  determined  and  caanges  in  the  times  of  Lh^,  year 
that  students  were  tested. 

In  the  1988  main  assessment,  the  respective  modal  grades  for  ages  9,  13, 
and  17  are  4,  8,  and  12.     Student  age  for  all  three  cohorts  was  determined  on 
a  calendar-year  basis;  all  students  were  tested  at  the  same  times  of  the  year. 
The  1988  bridge  samples,  by  definition  designed  to  match  previous  assessment 
characteristics,  sampled  varying  student  cohorts. 

The  1988  main  sample  included  many  students  in  each  cohort  who  were  both 
age-eligible  (age  9,  13,  or  17)  and  grade-eligible  (attendin^  respectively 
grade  4,  8,  or  12).    However,  because  NAE?  collected  data  by  grade  or  a^**, 
each  cohort  also  included  students  who  were  age-eligible  but  not  ir.  cne  modal 
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grade,  and  students  who  were  grade-eligible  but  not  of  the  modal  age  (the  age 
of  most  students  attending  the  particular  grade). 

For  each  1988  sample,  results  for  students  in  a  particular  grade  can  be 
selected  using  (1)  the  variable  DGRADE,  the  student's  actual  grade  at  time  of 
testing,  on  the  student  file,  or  (2)  the  student  file  variable  MODGRD  (setting 
MODGRD  to  a  value  of  2  will  select  those  students  who  are  in  the  modal  grade). 

DAGE,  MODAGE  (Student  age) 

For  the  1988  main  sample,  student  age  for  all  three  grade/age  cohorts 
was  calculated  as  of  Decem>^er  31,  1987.  Because  NAEP  collected  data  by  grade 
or  age,  each  main  sample  student  cohort  includes  students  who  were  both  agt.- 
eligible  and  grade -eligible,  students  who  were  age-  eligible  but  not  in  the 
modal  grade,  and  students  who  were  grade-eligible  but  not  of  the  modal  age. 
The  modal  ages  for  each  grade/age  cohort  in  the  main  sample  were  defined  by 
the  following  birth  dates,  based  on  a  calendar  year: 

Age  9:  born  between  January  1  and  December  31,  1978 
Age  13:  born  between  January  1  and  December  31,  1974 
Age  17:  born  between  January  1  and  December  31,  1970 

For  the  bridge  to  1982,  samples  of  age  13  and  age  17  were  selected  based 
on  the  following  birth  dates: 

Age  13:    born  between  January  1  and  December  31,  1974 

Age  17:    born  between  October  1,  1970  and  September  30,  1971 

The  sample  of  grade  4/age  9,  grade  8/age  13,  and  grade  11/age  17 
students  for  the  bridge  to  1984  were  selected  according  to  the  following  birth 
dates: 

Age  9:  born  between  January  1  and  December  31,  1978 
Age  13:  born  between  January  1  and  December  31,  1974 
Age  17:    born  between  October  1,  1970  and  September  30,  1971 

The  samples  of  ages  9  and  13  and  grade  11/age  17  students  for  the  bridge 
to  1986  were  selected  according  to  the  following  birth  dates: 

Age  9:  born  between  January  1  and  December  31,  1978 
Age  13:  born  between  January  1  and  December  31,  1":^74 
Age  17:    born  between  October  1,  1970  and  September  30,  1971 

For  all  1988  sampl«*s,  results  for  students  in  a  particul  r  age  can  be 
selected  u.sing  (1)  the  variable  DAGE,  the  student's  actual  age  at  J.me  of 
testing,  on  the  .student  file  or  (2)  the  student  file  variable  MODaGE  (setting 
MODAGE  to  a  value  of  2  will  select  those  students  who  are  of  the  modal  age). 
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COMPOSITE  AND  DERIVED  COMMON  BACKGROUND  VARIABLES 

Several  NAEP  variables  are  formed  from  the  systematic  combination  of 
response  values  for  one  or  more  common  background  items  (items  given  to  e\er_y 
student  concerning  subjects  ruch  as  materials  in  the  home,  languages  spoken, 
hours  spent  watching  television,  and  after-school  activities). 

The  derived  background  variables  maximize  use  of  the  data,  incorporate  a 
larger  segment  of  the  population,  and  save  analysis  costs  by  grouping  items 
that  measure  similar  characteristics  into  one  variable.     Some  of  these 
variables  were  used  in  the  NAEP  subject  area  reports.    The  derivation  of  each 
of  these  variables  is  explained  in  the  following  sections. 


HOMEENV  (Home  Environment — ^Articles  in  the  Home) 

For  all  three  age  classes,  the  variable  HOMEENV  was  created  from  the 
responses  to  background  items  B000901  through  B000905  concerning  articles 
found  in  the  student's  home  (newspaper,  dictionary,  encyclopedia,  more  than  25 
books,  and  magazines).    The  values  for  this  variable  were  derived  as  follows. 

1  0-3  ARTICLES      The  student  responded  to  at  least  tl  ree  items  and  answered 

YES  to  three  or  fewer. 

2  4  ARTICLES  The  student  answered  YES  to  four  items. 
:    5  ARTICLES         The  student  answered  YES  to  five  items. 

8    NO  RESPONSE       The  student  answered  fewer  than  three  items. 


H0MEEN2  (Home  Environment — ^Articles  [of  hj  in  the  Home) 

For  all  three  age  classes,  the  variable  H0MEEN2  was  created  from  ^e 
responses  to  backgroand  items  B000901,  B000903,  B000904,  and  B000905 
concerr^ing  articles  found  in  the  student's  home  (newspaper,  encyclopedic,  more 
than  25  books,  and  magazines).    The  values  for  this  variable  were  derived  as 
follows: 

1  0-2  ARTICLES      The  student  responded  to  at  least  two  items  and  answered  YES 

to  two  or  fewer. 

2  3  ARTICLES         The  student  answered  YES  to  throe  5tems. 

3  4  ARTICLED         The  student  answered  YES  to  four  items. 

8    NO  RESPONSE       The  student  answered  -^wer  than  two  items. 
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NCOMP  (Number  of  Computer  Science  Courses  Teken) 

For  age  class  17,  NCOM?  was  created  from  responses  to  items  B005312  and 
B005313  concerning  the  student's  coursework  in  computer  science.    The  values 
for  NCOMP  were  derived  as  follows: 


1 

0 

Thr 

student 

answered  HAVE  NOT  to  both  courses. 

2 

1 

The 

student 

answered  HAVE  to  one  course. 

3 

2 

The 

student 

answered  YES  to  both  courses. 

8 

NO  RESPONSE 

The 

student 

did  not  respond  to  one  or  both  items. 

9 

MULT.  &  OUT-OF-RANGE 

The 

student 

filled  in  more  than  one  oval  for  both 

items. 


NMATH  ^Highest  Level  of  Mathematics  Courses  Taken) 

For  age  class  17,  NMATH  was  created  from  responses  to  items  B005301 
thi-ough  B005307  concerning  the  student's  coursework  in  mathematics.  The 
values  for  NMATH  were  derived  as  follows: 


1  GEN.  i4ATH  OR 
PRE -ALGEBRA 

2  ALGEBRA 


3  GEOMETRY 

4  ALGEBRA  2 


The  student  answered  HAVE  rOT  to  all  items  or  HAVE  to 
B005301  or  B005302  and  HAVE  ^^CT  to  all  others. 

The  student  answered  HAVE  to  B005303  and  HAVE  NOT  to 
B005304,  B005305,  B005306,  and  B005307. 

The  student  answered  HAVE  to  both  Bo05303  and  B005305  and 
HAVE  NOT  to  B005304,  B005306,  and  B0053O7. 

The  student  answered  HAVE  to  B00530  or  B005306  but  HAVE  NOT 
to  B005307. 


CALCULUS  The  student  answered  HAVE  to  B005307. 

6    SOMETHING  ELSF  Any  other  response  combination 
8    NO  RESPONSE       The  student  did  not  respond  to  any  item. 


NSCI  (Highest  Level  of  Science  Courses  Taken) 

For  age  class  17,  NSCI  was  created  frora  responses  to  items  B005308 
through  B005311,  which  concerned  the  studert's  coursework  in  science.  The 
values  for  NSCI  were  derived  as  follows: 

1    NO  BIOLOGY         The  student  answered  HAVE  NOT  to  all  items  or  HAVE  to 
B005308  and  other  than  HAVE  to  all  other  items. 
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2  BIOLOGY 


3  CHEMISTRY 


The  student  answered  HAVE  to  B005309  and  other  than  HAVE  to 
both  B005310  and  B005311. 

The  student  answered  HAVE  to  both  B005309  and    *^M53lO  and 
other  than  HAVE  to  B005311. 


4  PHYSICS  The  student  answered  HAVE  to  B005309,  B005310,  and  B005311. 

5  SOMETHING  ELSE  Any  other  response  combination 

8    NO  RESiONSE       The  student  answered  none  of  the  items. 


NSIBS  (Number  of  Siblings) 

For  a^l  age  classes,  NSIBS  was  created  from  responses  to  item  B005801 
(How  many  brothers  and  sisters  do  you  have?).    The  values  for  NSIBS  were 
derived  as  follows: 


1  NONE 

2  1 

3  2 

4  3 

5  4  OR  MORE 

8  NO  RESP0N1.E 

9  MULT. 


The  student  answered  NONE. 
The  student  answered  ONE. 
The  student  answered  TWO. 
The  student  answered  THREE. 

The  student  answered  FOUR,  FIVE,  or  SIX  OR  MORE. 

The  student  did  not  respond. 

The  student  filled  in  more  than  one  oval. 


PARWK4,  PARWK5  (Economic  Support — ^Which  Parents  Work) 

For  age  class  9,  PARWK4  was  created  from  responses  to  items  B005901  and 
B006101,  which  asked  if  the  student's  mother  (or  stepmother)  and  father  (or 
stepfather)  worked  for  pay.    The  values  for  PARWK4  were  derived  as  follows. 

1    BOTH  MOTHER  &  FATHER  The  student  answered  YES  to  both  B005901  and  B006101. 


2  FATHER  ONLY 

3  MOTHER  ONLY 

4  SOMETHING  ELSE 
8  NO  RESPONSE 


The  student  answered  NO  or  DON'T  LIVE  WITH  to  B005901 
and  YES  to  B006101. 

The  student  answered  NO  or  DON'T  LIVE  WITH  to  B006101 
and  YES  to  B005901. 

Any  other  combination  of  responses 

The  student  did   iOt  respond  to  one  or  both  items. 
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9  MULT. 


The  student  filled  in  more  than  one  oval  for  both 
items 


For  age  classes  13  and  17,  PARWK5  was  created  from  responrjes  to  items 
B006001  and  B006201,  which  asked  if  the  student's  mother  (or  ste,, mother)  and 
father  or  (stepfather)  worked  part-time  or  full-time  for  pa>  .    T.ie  values  for 
PARWK5  were  derived  as  follows: 

1  BOTH  FULL  TIME  The  student  answered  YFS.  FULL-TIME  to  both  B006001  and 

B006201. 

2  BOTH  WORK  SOME  The  suudent  answered  YES,  FULL-TIME  or  YES,  PART-TIME  to 

BOO  201  and  YES,  FULL-TIME  or  YES,  PART-TIME  to  B006001. 

3  FATHER  ONLY       The  student  answered  YES,  FULL-TIME  or  YES,  PART-TIME  to 

B006201  and  NO  or  DON'T  LIVE  WITH  to  B006001. 

4  MOTHER  ONLY       The  student:  answered  YES,  FULL-TIME  or  YES,  PART-TIME  to 

B006001  and  NO  or  DON'T  LIVE  WITH  to  B006201. 

5  SOMETHING  ELSE  Any  other  combination  of  responses 

8  NO  RESPONSE       The  student  did  not  respond  to  one  or  both  items. 

9  MULT.  The  student  filled  in  more  than  one  oval  frr  both  items. 


RACLANG         (By  Race,  Language  Other  than  English  is  Spoken  by  Student  in 
Home) 

For  all  age  classes,  the*  variable  RACLANG  was  created  from  responses  to 
item  B003401  (Do  you  speak  a  language  other  than  English  in  your  home?)  and 
the  imputed  race/ethnicity  variable  DRACE.    The  values  for  PJVCLANG  were 
derived  as  follows: 


1  HISPANIC 

2  ASIAN 

3  OTHER  RACE 

4  NO 

8  NO  RESPONSE 

9  MULT. 


The  student  answered  YES  to  B003401,  ano  DRACE  was  HISPANIC. 

The  student  answered  YES  to  B003401,  and  DRACE  was  ASIAN. 

The  student  answered  YES  to  B003401,  and  DRACE  was  anything 
other  than  HISPANIC  or  ASIAN. 

The  student  answered  NO  to  B003401. 

The  student  did  not  respond  to  B00340i. 

The  student  filled  in  more  than  one  oval  for  B003401. 
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ElACOFTN         (By  Race,  How  Often  Language  Other  than  English  is  Spoken  by 
Others  in  Home) 


For  age  class  9,  the  variable  RACOFTN  was  created  from  item  B003201  (How 
often  do  the  people  in  your  home  speak  a  language  other  than  English?)  and  the 
Imputed  race/ethnicity  variable  DRACE.  The  values  for  RACOFTN  for  age  clas^  9 
were  derived  as  follows: 


1  HISPANIC  The  student  answered  SOMETIMES  or  ALWAYS  to  BOO'^'^Ol,  and 

DRACE  was  HISPANIC. 

2  ASIAN  The  student  answered  SOMETIMES  or  ALWAYS  to  B003201,  and 

DRACE  was  ASIAN. 

3  OTHER  RACE         The  student  answered  SOMETIMES  or  ALWAYS  to  B0032O1,  and 

DRACiS  was  not  HISPANIC  or  ASIAN. 


4    NEVER  The  student  answered  NEVER  to  B003201. 

8  NO  RESPONSE       The  student  d^d  not  respond  to  B003201. 

9  MULT.  The  student  filled  in  more  than  one  oval  for  B003201. 


For  age  classes  13  and  17,  the  variable  RACOFTN  was  created  from  item 
B003301  (How  often  do  the  people  in  your  home  speak  a  language  other  than 
English?)  and  the  imputed  race/ethnicity  variable  DRACE.    The  values  for 
RACOFTN  for  these  age  clas.<;es  were  derived  as  follows: 


1    HISPANIC  The  student  answered  ABOUT  HALF  THE  TIME,  MOST  OF  THE  TIME, 

or  ALWAYS  to  B003301,  and  DRACE  was  HISPANIC. 


2  ASIAN 


The  student  answered  ABOUT  HALF  THE  TIME,  MOST  OF  THE  TIME, 
or  ALWAYS  to  B003301,  and  DRACE  was  ASIAN. 


3    OTHER  RACE 


The  student  answered  ABOUT  HALF  THE  TIME,  MOST  C  THE  TIME, 
or  ALWAYS  to  B003301,  and  DRACE  was  not  HISPANIC    r  ASIAN. 


NEVER 


The  student  answered  NEVER  or  OCCASIONALLY  to  B0O3301. 


8  NO  RESPONSE       The  student  did  not  respond  to  B003301. 

9  MULT.  The  student  filled  in  more  than  oue  oval  for  B003301. 


SINGLEP  (How  Many  Parents  Live  at  Home) 

For  all  three  age  classes,  SINGLEP  was  created  from  items  B005601  and 
B005701,  which  asked  whether  the  student's  mother  (or  stepmother)  and  father 
(or  stepfather)  lived  at  home  with  the  student.     The  values  for  SINGLEP  were 
derived  as  follows: 

1    2  PARENTS  AT  HOME       The  student  answered  \ES  to  both  items. 
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2  1  PARENT  AT  HOME 

3  NEITHER  AT  HOME 

8  NO  RESPONSE 

9  MULT, 


The  s.uden»  an  wered  YF:  to  B005601  and  NO  to  B005701, 
or  YES  to  B005701  and  NO  to  B005601. 

The  student  answered  NO  to  both  items. 

The  student  did  not  respond  to  one  or  both  items. 

The  ctudept  filled  in  more  than  one  oval  for  one  or 
both  items. 


SINGPB  (Which  Parents  Live  at  Home) 

For  all  age  classes,  SINGPB  was  created  from  items  B005601  and  B005701 
concerning  whether  the  student's  rcotner  (oi.  stepmother)  and  father  (or 
stepfather)  live  at  home  with  the  student.    The  values  for  SINGPB  were  derived 


as 

follows : 

1 

MOTHER  AND  FATHER 

The 

student 

answered  YES  to  both  items. 

2 

FATHER  ONLY 

The 

student 

c.iowered  YES  to  B005701  and  NO  to  B005601. 

3 

MOTHER  ONLY 

The 

student 

answered  YES  to  B005601  and  NO  to  B005701. 

4 

NEITHER 

The 

student 

answered  NO  tc  both  items. 

8 

NO  RESPONSE 

The 

student 

did  not  respond  to  one  or  both  items. 

9 

MULT. 

The  student 
both  items. 

filled  in  more  than  one  oval  for  one  or 
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SINGFEM  (Single  Female  Head  of  Household— Working) 

SINGFEM  was  created  from  the  variables  SINGPB  and  PARWK5  (PARV7K4  for  age 
class  9)  concerning,  respectively,  which  parents  live  at  home  a-id  which 
parents  work.     The  values  for  SINGFEM  were  derived  as  follows: 


1  YES 


2  NO 


SINGPB  was  3  and  PARWK4  was  3  (age  class  9);  SINGPB  was  3 

and  PARWK5  was  4  (other  age  classes). 

SINGPB  was  3  and  PARWK4  was  4  (age  class  9);  SINGPB  was  3 

and  PARWK5  was  5  (other  age  classes). 


3    SOMETHING  ELSE  Any  ocher  response  combination. 

8    NO  RESPONSE       No    ^lue  was  derived  for  one  or  both  variables  because  of 
nonresponse. 


9  MULT. 


No  value  was  derived  for  one  or  both  variables  because  of 
multiple  respor.se. 
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TVWATCH  (Amount  of  Television  Viewing  Each  Day) 

For  all  three  age  classes,  the  variable  TVWATCH  was  created  from  item 
B001801  (How  much  television  do  you  usually  watch  tach  day?).    The  values  for 
TVWATCH  were  derived  as  follows: 


1  0-2  HOURS 

2  3-5  HOURS 

3  6  OR  MORE  HOURS 

8  NO  RESPONSE 

9  MULT. 


The  student  answered  NONE,  ONE  HOUR  OR  LESS,  or  TWO 
HOURS  to  B001801. 

The  student  answered  THREE  HOURS,  FOUR  HOURS,  or  FIVE 
HOURS  to  B00180I. 

The  student  answered  SIX  HOURS  OR  MORE  to  B001801. 

The  student  did  not  respond. 

The  student  filled  in  more  than  one  oval. 
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SUBJECT-SPECIFIC  COMPOSITE  AND  DERIVED  REPORTING  V/  JABLES 


Reading  Trend  Reporting  ^^ariables 

RTENGAG    (Engagement  in  Reading- related  Activities) 

For  ages  9,  13,  and  17,  RTENGAG  was  created  from  items  S003502,  S003503, 
S003504,  and  S003506,  which  asked  how  often  students  engaged  in  certain 
activities  related  to  reading.    The  values  for  these  items  were  recoded  as 
follows : 

Daily,  weekly,  monthly,  yearly  -  Ever 

Never  -  Never 

The  recoded  responses  for  all  four  items  were  then  combined  as  follows; 

Never  for  all  four  items  or 

ever  for  one  item  -  0-1  activiry 

Ever  for  two  items  -  2  activities 

Ever  for  three  items  -  3  activities 

Ever  for  four  items  -  4  activities 


RTHOME    (Extent  of  Reading  in  the  Home) 

For  ages  13  and  17,  RTHOME  was  created  from  items  S004501,  S004502,  and 
S004503,  which  asked  atudents  how  often  the  people  they  live  with  read 
newspapers,  magazines,  and  books.    The  items  were  recoded  as  follows: 

Daily  -  5 

Weekly  -  4 

Monthly  -  3 

Yearly  -  2 

Never  -  1 

The  average  of  the  three  recoded  responses  was  then  assigned  one  of  the 
following  labels: 

1.0    -  3.5    -  Low  (Never/Yearly/Monthly) 
3.51-4.5    -  Medium  (Weekly) 
4.51  -  5.0    -  High  (Daily) 
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RTPLEAS    (Reading  of  Books,  Newspapers,  and  Magazines) 

For  ages  9,  13,  and  17,  RTPLEAS  was  created  from  items  S004301,  S004304, 
and  S004305,  which  asked  students  how  often  they  read  stories  or  novels, 
newspapers  or  magazines.    The  items  were  receded  as  follovs: 


Daily 

Weekly 

Monthly 

Yearly 

Never 


-  5 

-  4 

-  3 

-  2 

-  1 


The  ave*.age  of  the  three  recoded  responses  was  then  assigned  one  of  the 
following  labels: 

1.0    -  3.5    -  Tx)w  (Never/Yearly/Monthly) 
3.51-4.5    -  Medium  (Weekly) 
4.51-5.0    -  High  (Daily) 


Writing  Cross-sectional  Reporting  Variables 

WENJOY    (Reported  Frequency  of  Enjoyment  of  Writing) 

For  grades  4,  8,  and  12,  item  S001201  was  recoded  as: 

Almost  always  -  5 

More  than  half  the  time       -  4 

Half  the  time  -  3 

Less  than  half  the  time       -  2 

Never  -  1 


WINST2    (Frequency  of  Writing  Process  Instruction) 

For  grades  8  and  12,  items  S000606,  S000608,  S000609,  and  S000610  were 
recoded  as: 


Almost  always  -  5 

More  than  half  the  time  -  4 

Half  the  time  -  3 

Less  than  half  the  time  -  2 

Kever  -  1 

The  average  response  t.o  these  four  items  was  then  rounded  to  the  nearest 

integer  and  the  same  olive  categories  v;ere  used  to  classify  the  variable. 
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WPLANNG  (Reported  Frequency  of  Planning) 


For  grade  12,  items  S000901  and  S000902  were  tecoded  as: 


Almost  always 

More  than  half  the  time 

Half  the  titie 

Less  than  half  the  time 

Never 


J 
4 
3 
2 
1 


The  average  response  to  these  two  items  was  then  rounded  to  the  nearest 
integer  and  the  same  five  categories  were  used  to  classify  the  variable. 

WREVISE  (Reported  Frequency  of  Revisfng  and  Editing) 


For  grade  12,  items  S001302,  S001303,  S001304,  S001308,  and  S001310  were 


The  average  re.  onse  to  these  five  items  was  then  rounded  to  the  nearest 
integer  and  the  same  five  categories  were  used  to  classify  the  variable. 

Writing  Trend  Reporting  Variables 

WVALUE  (Overall  Value  Placed  on  Writing) 

For  grades  8  and  12,  items  S001501,  S001502,  S001503,  S001504,  S001401, 
and  S001402  were  recoded  as  follows: 


recoded  as: 


Almost  always 

More  than  half  the  time 

Iialf  the  time 

Less  than  half  the  time 

Never 


5 
4 
3 
2 
1 


Almost  always 

More  than  half  the  time 

Half  the  time 

Less  than  half  the  time 

Never 


5 
4 
3 
2 
1 


The  average 
categories: 


response  to  these  six  items  was  then  con^'^^ted  to  three 


Low 
Medium 


High 


-  less  than  3.5 

-  between  3.5  and  4.5 

-  greater  than  or  equal  to  4.5 
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WATTID  (General  Attitude  Toward  Writing  Composites) 


For  grades  4,  8,  and  11,  items  S001201,  S001202,  S001204,  S001205,  and 
S001207  were  recoded  as  fellows: 

Aliaost  always  -  5 

More  than  half  the  time  -  4 

Half  the  time  -  3 

Less  than  half  the  time  -  2 

Never  -  1 

The  average  response  to  these  five  items  was  then  rounded  and  c  averted  into 
three  categories: 

Low  -  less  than  2.5 

Medium  -  between  2.5  and  3.5 

High  -  greater  than  or  equal  to  3.i» 


WFJSE  (Personal  and  Social  Uses  of  Writing) 

For  grades  8  and  11,  items  S001601,  S001602,  S001603,  S001604,  and 
S001609  were  recoded  as  follows: 

Daily  -  4 

Weekly  -  3 

Monthly  -  2 
Never  -  1 

The  average  response  to  these  five  items  was  then  rounded  and  converted  to 
three  categories: 

Low  -  less  than  1.5 

Medium  -  between  1.5  and  2.5 

High  -  greater  than  or  equal  to  2.5 


WREVED  (Revising  Strategies) 

For  grades  4,  8,  and  11,  items  S002002  and  S002003  were  first  recoded 

as: 

Yes  -  1 
No  -5 


ERLC 
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Then  Items  S000904,  S000906,  and  S000907  were  receded  as: 

Almost  always  -  5 

More  than  half  the  time  «  4 

Half  the  time  -  3 

Less  than  half  the  time  -  2 

Never  -  1 

The  average  response  to  these  five  items  was  then  rounded  and  converted  uo 
three  categories: 

Low  -  less  than  2.5 

Medium  -  between  2.5  and  3.5 

High  -  greater  than  or  equal  to  3.5 


WTFEED  (Teachers'  Feedback  on  Writing) 

For  grades  4,  8,  and  11,  items  S002501,  S002502,  S002503,  S002504, 
S002505,  and  S002506  were  recoded  as: 

Aljnost  always  -  5 

More  than  half  the  time  -  4 

Half  the  time  -  3 

Less  than  half  the  time  -  2 

Never  «  1 

The  average  respons'*  to  these  five  items  was  then  roanded  and  converted  into 
three  categories: 

Low  -  less  than  3.5 

Medium  -  between  3.5  and  4.5 

High  -  greater  than  or  equal  to  4.5 


Civics  Cross-sectional  Reporting  Variables 

PTOPICl  (Variety  of  Topics  Studied) 

For  grade  4/age  9,  items  P800601,  P800701,  P800801,  P800:^01,  P801001, 
and  P801101  asked  students  to  inuicate  whether  they  had  studied  particular 
civics -related  topics  a  lot,  some,  or  not  at  all.     Each  of  the  six  items  was 
coded  as  follows: 

A  lot  -  3 

Some  -  2 

Not  at  all  -  1 
Missing       -  M 
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The  average  x  the  six  receded  variables  was  then  assigned  the  labels  and 
codes: 


2.5-  3  -  A  lot     -  3 

1.6-  2.4  -  Some  -  2 
1-1.5  -  None       -  1 

M  for  all  ~  Missing  -  0  or  M 

The  average  was  across  all  items  that  had  a  response.     PTOPICl  appears  in  the 
data  sets  on  the  public-use  data  tapes  fcr  students  in  the  focused-BIB 
samples . 

PSTUDY2  (Whether  Civics  Was  Studied) 

For  grade  8/age  13,  items  j?800101  to  P800104  asked  students  whether  they 
bad  studied  American  government  or  civics  in  grades  5  to  8,  respectively. 
Each  of  the  four  items  was  coded  as: 

Yes  -  1 

No  ~  0 

I  don' t  know  -  0 
Missing  —  M 

The  sum  of  the  four  lecoded  vrriables  was  then  assigned  the  following  labels 
and  codes: 

1-4  -  Studied  at  least  1  year  -  1 

0  -  Not  studied  -  2 

M  for  all    -  Missing  -  0  or  M 

The  sum  was  across  all  items  thar  had  a  response.     PSTUDY2  appears  in  the  data 
sets  on  the  public-use  data  tapes  for  student"  in  both  the  focused-BIB  and 
intercor relation  samples. 

PT0PIC2  (Variety  of  Topics  Studied  -  Mean  of  10  Questions) 

For  grade  8/age  13,  items  P801201  to  P801210  asked  students  to  indicate 
whether  they  had  studiad  particular  civics-related  topics  a  lot,  some  or  not 
at  all.     Each  of  the  ten  it'2ms  was  ::oded  as  follows: 


A  lot  -  3 

Some  -  2 

Not  at  all  -  1 
Missing       -  M 


The  average  of  the 
labels  and  codes: 


ten  recoded  variables  was  then  assigned  the  following 


2.5-  3  -  A  Lot     -  3 

1.6-  2.4  -  Some  -  2 
1-1.5  -  None       -  1 

M  for  all  -  Missing  -  0  or  M 

The  average  was  across  all  items  that  had  a  response.    PT0PIC2  appears  in  the 
datA  sets  on  the  public-use  data  tapei.  for  students  in  the  focused-BIB 
sampi.es . 

PSTUDY3  (Whether  Civics  Was  Studied) 

For  grade  12/age  17,  items  P800201  to  P800204  asked  students  whether 
they  had  studied  American  government  or  civics  in  grades  9  to  12, 
respectively.    Each  of  che  four  items  was  coded  as: 

Yes  -  1 

No  -  0 

I  don't  know  -  0 
Missing  -  M 

The  sum  of  the  four  recoded  variables  was  then  assigned  the  following  labels 
and  codes: 

1-4  -  Scudied  at  least  1  year  -  1 

0  -  Not  studied  -  2 

M  for  all    -  Missing  ~  0  or  M 

The  sum  was  across  ail  items  that  had  a  response.    PSTUDY3  appears  in  the  data 
sets  on  tht  public-use  data  tapes  for  students  in  both  the  fccused-BIB  and 
intercorrelation  samples. 

PT0PIC3  (Variety  of  Topics  Studied) 

For  grade  12/age  17,  items  P801301  to  P801310  asked  students  to  indicate 
whether  they  ha^.  studied  particular  civics- related  topics  a  lot,  some,  or  not 
at  all.     Each  of  the  ten  items  was  coded  as  follows: 


A  lot  -  3 

Some  -  2 

Not  at  all  -  1 
Missing       -  M 
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The  average  of  the  ten  recoded  variables  was  then  assigned  the  following 
labels  and  codes: 


2.5-  3 

1.6-  2. A 
1-1.5 

M  for  all 


-  A  Lot    «  :j 

-  Some       -  2 

-  None       -  1 

-  Missing  -  0  or  M 


The  average  was  across  all  items  that  had  a  response.  PT0PIC3  appears  in  the 

data  sets  on  the  public -use  data  tapes  for  students  in  the  focused-BIB 
samples. 

PHMWK  (Civics  Homework) 

For  grade  12/age  17,  values  for  item  P801901  .^ere  assigned  the  following 
labels  and  codes: 


No  assignments 

None 

1 

Don't  do  assignment 

Don't  do 

2 

Less  than  1  hour 

Hr/less 

3 

1  hour 

Hr/less 

mm 

3 

2  hours 

2  hrs 

mm 

4 

3  hours 

3 /mo re 

mm 

5 

4  hours 

3/mcre 

5 

5  hours  or  more 

3 /mo re 

5 

Missing 

mm 

Missing 

0 

M 


Civics  Trend  Reporting  Variables 

PPARED  (Parent's  Education  -  2nd  Set  of  Categories) 

For  ages  13  and  17,  a  parents'  education  variable  was  derived  for  1:^82 
datL,  so  that  comparisons  with  1988  data  be  made  for  students  having  a  parent 
>.'ho  graduated  from  college.    This  variable  is  based  on  the  1982  variables 
PARED,  FCLGRAD,  and  MCLGRAD  (whether  the  father  or  mother  graduated  from 
college).    The  categories  for  PPARED  are  coded  as: 


1 
2 
3 


High  school 
Lgh  school  gi 
Post  high  school 

College  graduate 

Missing 


if  PARED  -  1  r<HS) 
High  school  graduate  if  PARED  -  2  (HS  Grad) 

if  PARED  -  3  (post-HS), 
and  MCLGRAD  -  2  (No) 
if  PARED  -  3  and  either  FCLGRAD 
or  MCLGRAD  -  1  (Yes) 
if  otherwise 


FCLGPJU)  -  2 (No), 
1  (Yes) 
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U>S.  History  Cross-sectional  Reporting  Variables 
HTOPICS         (Topics  studied) 


For  grade  4/age  9,  responses  to  items  H801101,  H801201,  H801301, 
H801401,  H801501,  H801601,  and  H801701  were  recoded  as  follows: 

A  lot  -  3 

Some  -  2 

Not  at  all  -  1 

The  average  of  the  seven  recoded  responses  ^f^as  then  transformed  to  form 
the  HTOPICS  vcriable  as  follows: 

>  2*5       A  lot  -  3 

>  1.5  but  <  2.5        Some  «  2 

:S  1.5       Not  at  all  -  1 

Insufficient  data    -  0 


HTAKEN  (United  States  history  studied  or  expected  to  study) 

For  grade  8/r*%e  13,  responses  to  items  H800701  to  H800704  were  collapsed 
as  follows: 


Yes  to  any  item  -  1 

Ail  "I  don't  know"  -  2 

No  "Yes"  but  at  least  one  •'No"    -  3 

Insufficient  data  «  0 


HTOPICS         (Topics  studied) 

For  grade  8/age  13,  responses  to  items  H801801  to  H801811  were  recoded 
as  follows: 

A  lot  -  3 

Some  -  2 

Not  at  all  «  1 

The  average  of  the  li  recoded  responses  was  then  transformed  to  form  the 
HTOPICS  variable  as  follows: 

>  2.5      A  lot  -  3 

>  1.5  but  <  2.5      Some  -  2 

^1,5      Not  at  all  -  1 

Insufficient  data  -  0 
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HTAKEN  (United  States  history  studied  or  expected  to  study) 

For  grade  12/age  17,  responses  to  items  H800801  to  H800804  were 
collapsed  as  follows: 

Any  Yes  -  1 

All  "I  don't  know"  -  2 

No  "Yes"  but  at  least  one  "No"  ~  3 

Insufficient  data  -  0 


HTOPICS         (Topics  studied) 

For  grade  12/age  17,  responses  to  Items  H801901  to  H801913  were  receded 
as  follows: 

A  lot  -  3 

Some  -  2 

Not  at  all  -  1 

The  average  of  the  13  recoded  responses  was  then  transformed  to  form  the 
HTOPICS  variable  as  follows: 

^  2.5      A  lot  -  3 

>  1,5  but  <  2,5      Some  -  2 

^  1,5      Not  at  all  -  1 

Insufficient  data  -  0 


Geography  Cross-sectional  Reporting  Variables  (Grade  12/Age  17) 

GSKILLS         (Using  the  skills  and  tooL<5  of  geography) 

Responses  to  items  G800201.  G800202,  and  G800203  were  recoded  as 
follows: 

A  lot  -  3 

Some  -  2 

Very  little  -  1 

Not  at  all  -  0 

The  average  of  the  three  recoded  responses  was  then  transformed  to  form 
the.  GSKILLS  variable  as  follows: 

^  2,5      A  lot  -  3 

1,5  to  <  2.5      Some  -  2 

<  1.5     Very  little/Not  at  all  -  1 

Insufficient  data  «  0 


GLOCAT  (Knowing  locations) 


Responses  to  items  G800204,  G800205,  G000208,  and  G800209  were  racoded 
as  follows: 

A  lot  -  3 

Some  -  2 

Very  little  -  1 

Not  at  all  -  0 

The  average  of  the  four  recoded  responses  was  then  transformed  to  form 
the  GLOCAT  variable  as  follows: 

^  2.5     A  lot  -  3 

1.5  to  <  2.5      Some  -  2 

<  1.5     Very  little,/Not  at  all    -  1 

Insufficient  data  -  0 


GPHYSIC         (Understanding  physical  geography) 

Responses  i:o  items  GG00206  and  G800207  were  recoded  as  follows: 

A  lot:  -  3 

Some  -  2 

Very  little  -  1 

Not  at  all  -  0 

The  average  of  the  two  rec  ^ded  responses    as  then  transformed  to  form 
the  GPHYSir      "iable  as  follows: 

^.5     A  lot  -  3 

1.5  to  <  2.5      Some  -  2 

<  1.5      Very  little/Not  at  all  -  1 

Insufficient  data  -  0 

GCULT  (Understanding  cultural  geography) 

Responses  to  items  G800210  to  ^800215  were  recoded  as  follows: 

A  lot  "3 

Some  -  2 

Very  little  -  1 

Noc  at  all  -  0 
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The  average  of  the  six  recoded  responses  was  then  transformed  to  form 
the  GCULT  variable  as  follows: 


^  2.5     A  lot  -  3 

1.5  to  <  2.5      fome  -  2 

<  1.5     Very  little/Not  at  all  -  1 

Insufficient  data  -  0 


GTAKEN  (Geography  courses  taken  or  expected  to  take) 

Responses  to  items  G800301  to  G800304  were  collapsed  as  follows: 

Yes  to  any  item  «  1 

No  "Yes**  but  at  least  one  "NO"  -  2 

All  I  don't  know  -  3 

Insufficient  data  «  0 

GWORLD  (World  history  an^  geography  course  work  completed) 

Responses  zo  items  G800101  and  G800102  were  recoded  as  follows: 

Yes,  in  grade  9  -  1 

Yes,  in  grade  10  «  1 

Yes,  in  grade  11  -  1 

Yes,  in  grade  12  -  1 

No,  I  haven't  taken  -  2 


GEOUS 


rhe  two  items  were  then  collapsed  as  follows: 

Yes  to  either  item  -  1 

No  to  both  items  -  2 

Insufficient  data  -  0 


(U.S.  history  and  geography  course  work  completed) 


Responses  to  items  G800103,  G800104,  and  G800105  were  r  :oded  as 
follows: 


Yes,  in  grade  9  -  1 

Ye<;,  in  grade  10  -  1 

Yes,  in  grade  11  -  1 

Yes,  in  grade  12  -  1 

No,  I  haven't  taken  -  2 


The  three  items  were  then  collapsed  as  follows: 

Yes  to  either  iter  -  1 

No  to  both  items  - 
Insufficient  data  -  u 
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GPHY.E  (Physical  geography/earth  science  course  work  completed) 

Responses  to  item  G800106  were  recoded  as  follows: 


Yes,  in  grade  9  -  1 

Yes,  in  grade  10  -  1 

Yes,  in  grade  11  1 

Yes,  in  grade  12  -  1 

No,  I  haven't  taken  -  2 

The  item  was  then  collapsed  a^  follows: 

Yes  to  either  item  -  1 

No  to  both  items  -  2 

Insufficient  data  -  0 


GECONCM         (Economic,  political,  hunan,  cultural  and  urban  course  work 
completed) 

Responses  to  items  G8003  07,  G800108,  and  G800109  were  recoded  as 
follows : 

Yes,  in  grade  9  -  i 

Yes,  in  grade  10  "1 

Yes,  in  grade  11  -  1 

Yes,  in  grade  12  -  \ 

No,  1  hr'  m't  taken  -  2 

The  three  items  were  then  collapsed  as  follows: 

Yes  to  either  item  -  1 

No  to  both  items  ~  2 

Insufficient  data  -  0 


GTOPICS  (Average  of  GSKILLS,  GLOCAT,  GPHYSIC  and  GCULT) 

The  average  of  the  four  variables  GSKILLS,  GLOCAT,  GPHYSIC,  and  GCULT 
was  transformed  to  form  the  GTOPICS  variable  as  follows: 

>  2.5      A  lot  of  study  -  3 

>  1.5  but  <  2.5      Some  study  -  2 

0-1.5      Little  or  No  study  -  1 

Insufficient  data  -  0 
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Focusing  the  New  Design:  The  NAEP  1988  Technical  Report 
GLOSSARY  OF  TERMS 


anchoring*  The  process  of  characterizing  score 
levels  in  terms  of  predicted  observable 
behavior. 

assessment  session.  The  period  of  time  during 
which  a  NAEP  bookltU  is  administered  to  one 
or  more  individuals. 

average  response  method  (ARM).  A  regression- 
based  technique  to  predict  for  a  respondent 
the  conditional  distribution  of  an  average 
score  on  a  set  of  items  ^vcn  responses  to  at 
least  one  of  the  items  and  other  information. 

background  questionnaires.  The  instruments 
used  to  collect  information  about 
students'demographics  and  educational 
experiences. 

bias.  In  statistics,  the  difference  between  the 
expc..  icd  value  of  an  estimator  and  the 
population  parameter  being  estimated.  If  the 
average  value  of  the  estimator  over  all 
possible  samples  (the  estimator's  expected 
value)  equals  the  parameter  being  estim-^ted, 
the  estimator  is  said  to  be  unbiased; 
otherwise,  the  estimator  is  biased. 

BIB  (Balanced  Incomplete  Block)  splraling.  A 
complex  variant  of  multiple  matrix  sampling, 
in  which  items  are  administered  in  such  a  way 
that  each  pair  of  items  is  administered  to  a 
nationally  representative  sample  of 
respondents. 

BILOG.  A  computer  program  for  estimating  item 
parameters. 

block.  A  group  of  assessment  items  created  by 
dividing  the  item  pool  for  an  age/grade  into 
subsets.  Used  in  the  implementation  of  the 
BIB  spiral  sample  design. 

booklet  The  assessment  instrument  created  by 
combining  blocks  of  assessment  items. 

bridging.  An  administration  of  the  same  set  of 
exercises  under  two  different  conuitions  or  to 
two  different  populations  to  allow  a  statistical 


link  (bridge)  to  be  established  between  results 
under  the  different  circumstances. 

calibrate.  To  estimate  the  parameters  of  a  set  of 
items  from  responses  of  a  sample  of 
exambees. 

clustering.  The  process  of  forming  sampling  units 
as  groups  of  other  units. 

codebook.  A  formatted  printout  of  NAEP  data 
for  each  student,  excluded  student,  teacher, 
and  school  in  a  particular  grade/age. 

coefficient  or  variation.  The  raJo  of  the  standard 
deviation  of  an  estimate  to  the  value  of  the 
estimate. 

combined  ratio  estimator.  The  ratio  estimator 
resulting  frDm  firs*  estimating  the  numerator 
and  the  denominator  values  and  then  using 
the  quotient  of  these  as  the  estimate  of  the 
ratio* 

common  block.  A  group  of  background  items 
includeu  in  the  be^nning  of  every  assessment 
booklet. 

conditional  probability.  Probability  of  an  event, 
^ven  the  occurrence  of  another  event. 

conditioning  variables.  DemograpLlc.  and  other 
backgroimd  variables  characterizing  a 
respondent.  Used  in  construction  of  plausible 
values. 

cross-sectional  assessment  An  assessment  that 
provides  information  about  differences  in 
educational  performance  across  subgroups  of 
students.  It  docs  not  provide  information 
about  changes  in  students*  educational 
performance  across  time.  It  may,  however, 
provide  baseline  data  for  measuring  future 
trends. 

Current  Po^  elation  Survey.  A  household  sample 
survey  conducted  monthly  by  the  Bureau  of 
the  Census  to  provide  estimates  of 
employment,  unemployment,  and  other 
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characteristic  of  the  general  labor  force,  the 
population  as  a  whole,  an^  various  subgroups 
of  the  population. 

degntcs  of  freedom,  [of  a  variance  estimator] 
The  number  of  independent  pieces  of 
information  used  to  generate  a  variance 
estimate. 

derived  variables.  Subgroup  data  th?t  were  rot 
obtained  directly  from  assessment  responses, 
but  through  procedures  of  interpretation, 
classification,  or  calculation. 

design  effects.  The  ratio  of  the  variance  for  the 
sample  design  to  the  variance  for  a  simple 
random  sample  of  the  same  size. 

distractor.  An  incorrect  response  choice  included 
in  a  multiple-choice  item. 

excluded  student  questionnaire.  An  instrument 
completed  for  every  student  who  was  sampled 
but  excluded  from  the  assessment. 

excluded  students.  Sampled  students  determined 
by  the  school  to  be  unable  to  participate 
because  they  have  limited  En^ish  proficiency, 
are  mildly  mentally  retarded  (educable),  or 
are  functionally  disabled. 

expected  value.  The  average  of  the  sample 
estimates  given  by  an  estimator  over  all 
possible  samples.  If  the  estimator  is 
unbiased,  then  its  expected  value  will  equal 
the  population  value  being  estimated. 

field  test  A  pretest  of  items  to  obtain 

informal' -)n  regarding  clarity,  difficulty  levels, 
timing,  feasibility,  and  spedal  administrative 
situations',  performed  before  revldng  and 
selecting  items  to  be  used  in  the  assessment. 

focused-BIB  spiraling.  A  variation  of  BIB 
spiraiing  in  vAnch  items  are  administered  in 
such  a  way  that  each  pair  of  items  within  a 
subject  area  is  administered  to  a  national}' 
representative  sample  of  respondents. 

foils.  The  correct  and  in*,,  acct  response  choices 
included  in  a  multiple-choice  item. 

p*oup  effect.  The  difference  between  the  mean 
for  a  group  and  the  mean  for  the  nation. 

holistic  scoring.  A  method  of  evaluating  students* 
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writing  for  overall  fluency  in  responding  lo  a 
task. 

imputation.  Prediction  of  a  missing  value 
according  to  some  procedure,  using  a 
mathematical  model  in  combination  with 
available  information.  See  plausible  values. 

imputed  race/etbnidty.  The  race  or  ethnicity  of 
an  assessed  student,  as  derived  from  his  or 
her  responses  to  particular  common 
background  items.  A  NAEP  reporting 
subgroup. 

Item  response  theory  (IRT).  Test  analysis 

procedures  that  assume  a  mathematical  model 
for  the  probability  that  a  ^ven  examinee  will 
respond  correctly  to  a  ^ven  exercise. 

jackknife.  A  procedure  to  estimate  standard 
errors  of  percentages  and  other  statistics. 
Particularly  suited  to  complex  sample  designs. 

machine-readable  catalog.  Computer  processing 
control  information,  IRT  parameters,  foil 
codes,  and  labels  in  a  computer  Jablc 
format. 

m^Jor  strata.  Used  to  stratify  the  primary 

sampling  frame  within  each  region.  Involves 
stratification  by  size  of  community  and  degree 
of  ruralization  (SDOC). 

Metropolitan  statistical  area  (MSA).  An  area 
defined  by  the  federal  government  for  the 
purposes  of  presenting  jenerrl-furpose 
statistics  for  metropolitan  areas.  Typically,  an 
MSA  contains  a  city  with  a  population  of  at 
least  50,000  plus  adjacent  r^jeas. 

modal  age.  The  age  of  the  majority  of  a  group  of 
grade-eligible  students. 

modal  grade.  The  grade  attended  by  the  majority 
of  a  group  of  age-eli^ble  students. 

mode  of  administration.  The  method  by  which 
students  are  administered  assessment 
instruments.  Both  printed  and  tape-recorded 
administration  methods  are  used. 

multistage  sample  design.  Indicates  more  than 
one  rtagc  of  sampling.  An  example  of  tk 
.^tage  sampling:  1)  sample  of  counties 
(primary  sampling  units  or  PSUs);  2)  sample 
of  schools  within  each  sample  county; 
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3)  sample  of  students  within  each  sample 
school. 

multiple  matrix  sampling.  Samnling  plan  in 
which  difTcrent  samples  of  ^spondents  take 
dilTercKit  samples  of  items. 

NAEP  scales.  The  anchored  scilcs  common 
across  age/grade  levels  and  assessment  years 
used  to  report  NAEP  results. 

nonresponse*  The  failure  to  obtain  responses  or 
measurements  for  al^  imple  elements. 

nonsampiing  error.  A  general  term  applying  to 
all  .<U)urces  of  error  except  sampling  error. 
Includes  errors  from  defects  in  the  sampling 
frame,  response  or  measurement  error,  and 
mistakes  in  processing  the  data. 

objective*  A  desirable  education  goal  agreed  upon 
by  scholars  in  the  field,  educat-^rs,  and 
concerned  laypersons,  and  cstaolished  through 
the  consensus  approach. 

observed  race/ethnlclty.  Race  or  ethnidly  of  an 
assessed  student  as  perceived  by  the  exercise 
administrator. 

op?m-ended  response  Item.  A  non-multiple-choice 
item  that  requires  some  tj,    ^  ?  'ivrittea  or 
oral  response. 

oversi>.ztpli&g.  Deliberately  sampling  a  portion  of 
tie  population  at  a  hi^er  rate  than  the 
remainder  of  the  population. 

paced  tap^  The  audio  recording  that 
accompanies  some  booklets  to  assure 
uniformity  in  udministration.  R^rdcd 
instructions  prevent  reading  difficulties  from 
interfering  wiih  an  individual's  ability  to 
respond 

parental  educatioii.  The  level  of  education  of  the 
mother  and  father  of  an  assessed  student  as 
derived  from  the  student's  response  to  two 
assessment  items.  A  NAEP  reporting 
dubgronp. 

percent  correct  The  percent  of  a  target 

population  that  would  answering  a  pu'ticular 
exercise  correctly. 

piausibte  values.  Proficiency  values  drawn  at 
random  from  a  conditional  distribution  cf  a 


NAEP  respondent,  given  bis  or  her  response 
to  copinltive  exercises  and  a  specified  subset  of 
background  variables  (conditioning  variables) 
The  selection  of  a  plausible  value  is  a  Tjrm  of 
imputation. 

poststratiQcatlot?..  Classification  and  weighting  to 
correspond  to  external  values  of  selected 
sampling  units  by  a  set  of  strata  definitions 
after  the  sample  has  been  selected. 

primary  sampling  unit  (FSU).  The  basic 

geographic  sampling  unit  for  NAEP  Fi.acr  a 
sin^e  county  or  a  set  of  contiguous  counties. 

primary  trait  scoring.  A  method  of  evaluating 
students*  writing  for  effectiveness  in 
accomplishing  the  specific  goal  or  purpose 
each  writing  task. 

principal  questlonaaire.  A  data  collection  form 
^ven  to  school  principals  before  assessments, 
llie  principals  respond  to  questions 
concerning  enrollment,  size  anil  occupational 
composition  of  the  community,  etc. 

probability  sample.  A  sample  in  which  every 
element  of  the  population      a  known, 
nonzero  probability  of  be'  ,  :i. 

pseudnrepllcate.  The  value  of  a  statistic  based  on 
an  Jtered  sample.  Used  by  the  jackknife 
variance  estimator. 

public-use  data  tapes.  Computer  tapes  containi*  g 
respondent-level  cognitive,  background  and 
Siltitude,  and  demographic  data.  Availablr  "or 
»ise  by  rescarchf^rs  wishing  to  do  scconda* 
analyses  of  NAEP  cSta. 

QED.  Quality  Education  Data,  inc.  A  supplier  of 
lists  of  schools,  school  districts,  and  o'^hck 
school  data. 

random  variable.  A  variable  that  takes  on  any 
value  of  2  specified  ?ie  with  a  particular 
probability. 

re^oti.  One  of  four  geogrr^phic  areas  used  in 
gathering  and  re}>orting  ^^ata:  Northeast, 
Southeast,  Central,  and  West  (as  defined  b 
the  Office  Ot  Business  Economics,  U.b. 
Department  of  Commerce).  A  NAEP 
reporting  subgroup. 

reporting  subgroup  Groups  within  the  n.-^tional 
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population  for  which  NAEP  data  are 
repotted:  for  example,  gender,  race/ethnicily, 
grade,  age,  level  of  parental  education,  re^on, 
and  size  and  type  of  community. 

respondent  A  person  who  is  eligible  for  NAEP, 
is  m  the  sample,  and  responds  by  completing 
one  or  more  items  in  an  assessment  booklet. 

response  options.  In  a  multiple-chc:*^  question, 
alternatives  that  ^  m  be  se?caed  by  a 
respondent 

sample.  A  portion  of  a  population,  or  a  subset 
from  a  set  of  imits,  selected  by  some 
p/ooability  mechanism  for  the  purpose  of 
investigating  the  properties  of  the  population. 
NAEP  does  not  assess  an  entire  population 
but  rather  selects  a  representative  sample 
from  the  group  to  answer  assessment  items. 

sampling  error.  The  error  in  survey  estimates 
that  occurs  because  only  a  sample  of  ihe 
population  is  observed.  Measured  by 
sampling  standard  error. 

sampli^s  frame.  The  list  of  sampling  units  from 
wb'ich  the  sample  is  selected. 

sampling  weight  A  multiplicative  fact'^r  equal  to 
the  reciprocal  of  the  probability  of  a 
respondent  being  selected  for  assessment  with 
adjustment  for  nonresponse  and  perhaps  also 
for  poststratification.  The  sum  of  the  weights 
provides  an  estimate  of  the  number  of 
persons  m  the  po*»ulation  represented  by  a 
respondent  in  th'  sample. 

school  characteristics  and  policy  questionnaire. 
A  questionnaire  completed  for  each  school  by 
the  principal  or  other  official;  osed  to  gather 
information  concerning  school  administration, 
staffing  patterns,  curriculum,  and  student 
services. 

selection  probability.  The  chance  that  a 

panicular  sampling  unit  has  of  being  selected 
m  the  sample. 

simple  random  samp!e.  Process  for  selecting  n 
sampling  units  from  a  population  of  N 
sampling  units  so  that  each  sampling  unit  has 
an  equal  chance  of  being  in  the  sample  and 
every  combination  of  n  sampling  imits  has  the 
same  chance  of  being  in  the  sample  chosen. 
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size  and  type  of  community  (STOC).  One  of  the 
NAEP  reporting  subgroups,  dividing  the 
communities  m  the  nation  mto  seven  groups 
on  the  basis  of  size  and  other  characteristics. 

standard  error.  A  measure  of  sampling 
variability  and  measurement  error  for  a 
statistic.  Because  of  NAEP's  complex  sample 
design,  sampling  standard  errors  are 
estimated  by  jao^Iflng  the  samples  from 
first-stage  sample  estimates.  Standard  errors 
may  also  include  a  component  due  to  the 
error  of  measurement  of  individual  scores 
e&'Jsiaied  using  plausible  values. 

stratification.  The  division  of  a  population  mto 
parts,  called  strata. 

stratified  sample.  A  sample  selected  from  a 
population  that  has  been  stratified,  with  a 
sample  selected  independently  in  each 
stratum.  The  strata  are  defmed  for  the 
purpose  of  reducing  sampling  error. 

student  ID  .-lumben  A  unique  identification 
number  assigned  to  each  respondent  to 
preserve  his  or  her  anonymity.  NAEP  does 
not  record  the  names  of  any  lespondents. 

subject  area.  One  of  the  areas  assessed  by 

National  Assessment;  for  example,  art,  civics, 
computer  competefiCe,  geography,  literature, 
mathematics,  musi^  reading,  science,  U.^ 
history,  or  writing. 

systematic  sample  (syster  itic  random  sample). 
A  sample  selected  by  v  systematic  method;  for 
example,  when  units  are  selected  from  a  list 
at  equally  spaced  intervals. 

teacher  questionnaire.  A  questionnaire 

completed  by  selected  teachers  of  sample 
students;  used  to  gather  information 
concerning  years  of  teaching  experience, 
frequency  of  assignments,  teaching  materials 
used,  and  availability  and  use  of  computers. 

trend  assessment  An  assessment  based  on 

replicating  past  procedures  in  order  to  report 
changes  in  educational  achievement  across 
time. 

variance.  The  average  of  the  squared  deviations 
of  a  random  variable  from  the  expected  value 
of  the  variable.  The  variance  of  an  estimate 
is  the  squared  standard  error  of  the  estimate. 

71A 


REFERENCES  CITED  IN  TEXT 


ERIC 


715 


Fvwusing  the  New  Design:  The  NAEP  1988  Technical  Report 


REFERENCES  CITED  IN  TEXT 


Allen,  R.,  Bettis,  N.,  Kurfman,  D.,  MacDonald,  W.,  Mullis,  1.  V.  S..  &  Salter,  C.  (199^).  The  geography 

learning  of  higfi-school  seniors,  (No.  19  G  01)  Princeton,  NJ.  Educational  Testing  Service,  National 
Assessment  of  Educational  Progress. 

Anderson,  L.,  Jenkins,  L.  B.,  Leming,  J.,  MacDonald,  W.  B.,  Mullis,  I.  V.  S.,  Turner,  M.  J.,  &  Woostcr,  J.  S. 
(1990).  The  1988  civics  report  card  Trends  in  achievement  from  1976  to  1988  at  ages  13  and  17  and 
achievement  in  1988  at  grades  4,  8,  and  12.  Princeton,  NJ.  Educational  Testing  Service,  National 
Assessment  of  Educational  Progress. 

Applebec,  A.  N.,  Langer,  J.  A.,  &  Mullis,  I.  V.  S.  (1986).  The  writing  report  card.  Writing  achievement  in 

American  schools.  (No.  15-W-02)  Princeton,  NJ.  Educational  Testing  Service,  National  Assessment 
of  Educational  Progress. 

Applebee,  A.  N.,  Langer,  J.  A.,  &  Mullis,  I.  V.  S.  (1988).  Who  reads  best?  Factors  related  to  reading 

achievement  in  grades  3,  7,  and  11.  (No.  17-R-Ol)  Princeton,  NJ;  Educational  Testing  Service, 
National  Assessment  of  Educational  Progress. 

Applebee,  A.  N.,  Langer,  J.  A.,  &  Mullis,  I.  V.  S.  (1989).  Understanding  direct  writing  assessments. 
Reflections  on  a  South  Carolina  writing  study.  PrinCx-ton,  NJ:  Educational  Testing  Service. 

Applebee,  A.  N.,  Langer,  J.  A.,  MuUis,  L  V.  S.,  &  Jenkins,  L.  B.  (1990).  The  mting report  curd,  1984-88. 

Finding;^  from  the  Nation's  Report  Card.  (No.  19-W-Ul)  Princeton,  NJ.  Educational  Testing  Service, 
National  Assessment  of  Educational  Progresf.. 

Applebee,  A.  N.,  Langer,  J.  A.,  Mullis,  L  V.  S.,  Jenkins,  L.  B.,  &  Foertsch,  M.  A.  (1990).  Learning  to  write 
in  our  nation's  schools:  Instruction  and  achievement  in  1988  at  grades  4,  8,  and  12.  Princeton,  NJ. 
Educational  Testing  Service,  National  Assessment  of  Educational  Progress. 

B.  Jon,  A.  E.  (1387a).  The  NAEP  reading  scale.  In  A.  E.  Beaton,  Implementing  the  new  design.  The 

NAEP  1983-84  technical  report  (pp.  381-390).  (No.  15-TR-20)  Princeton,  NJ.  Educational  Testing 
Service,  National  Assessment  of  Educational  Progress. 

Beaton,  A.  E.  (1987b).  Imp'^ruenting  the  new  design.  The  NAEP  1983-84  technical  report.  (No.  15  TR  20) 
Princeton,  NJ:  Educational  Testing  Service,  National  Assessment  of  Educational  Progress. 

Beaton,  A.  E.  (198C).  Expanding  the  new  design:  The  NAEP  1985-86  technical  report.  (No.  17  TR  20) 
Princeton,  NJ.  Educational  Testing  Service,  National  Assessment  of  Educational  Progress. 

Beaton,  A.  E.,  &  Johnson,  E.  G.  (1990).  The  average  response  method  of  scaling.  Journal  of  Educational 
Statists,  15,  9-38.. 

BeaioP,  A.  E.,  &  Johnson,  E.  G.  (1987).  The  average  response  method  (ARM)  of  scaling.  In  A.  E.  Beaton, 
Implementing  tite  new  desist.  The  NAEP  1983-84  technical  report  (pp.  435-480).  (No.  15  TR  20) 
Princeton,  NJ.  Educational  Testing  Service,  National  Assessment  of  Educational  Progress. 

Beaton,  A.  E.,  &  Zwick,  R.  (1990).  The  effect  of  changes  in  the  National  Assessment.  Disentangling  the 
NAEP  1985-86  reading  anomaly.  (No.  17-TR-21)  Princeton,  NJ;  Educational  Testing  Service, 
National  Assessment  of  Educational  Progress. 


EMC 


717 


Bock,  R.  D.,  Gibbons,  R.  D.,  &  Muraki,  E.  (1985).  Full-information  item  factor  anafy^is.  (MRC  Report  No. 
85-1)  Chicago:  National  Opinion  Research  Center. 

Bock,  R.  D.,  Muraki,  E.,  &  Pfeiffenberger,  W.  (1988).  Item  pool  maintenance  in  the  presence  of  item 
parameter  drift.  Journal  of  Educational  Measurement^  25^  275-285. 

Breland,  H.  M.,  Camp,  R.,  Jone%  R.  J.,  Morris,  M.  M.,  &  Rock,  D.  A.  (1987).  Assessing  writing  skill. 
(Research  Monograph  Number  11)  New  York:  College  Entrance  Examination  Board. 

CdldwcU,  N.,  Moore,  D.,  &  Slobasky,  R.  (1989).  Report  on  field  operations  and  data  collection  activities 
NAEP  1988.  Rockville,  MD:  Westal,  Inc. 

Campbell,  A.  (1987).  Professional  scoring.  In  A.  E.  Boaion,  Implementing  the  new  desi^.  The  SAEP  1983 
84  technical  report  {pp.  nS'lM).  (No.  15-TR-20)  Princeton,  NJ.  Educational  Testing  Service, 
National  Assessment  of  Educational  Progress. 

Chambers,  J.  M.,  Qeveland,  W.  S.,  Kleiner,  B.,  &  Tukey,  P.  A.  (1983).  Graphical  methods  for  data  analysis, 
Boston:  Du>d)ury  Press. 

Cochran,  W.  F.  (1977).  Sampling  techniques.  New  York:  John  Wiley  &  Sons. 

Educational  Testing  Service.  (1987).  ETS  standards  for  quality  and  faimess.  Princeton,  NJ.  Educational 
Testing  Service. 

Educational  Testing  Service.  (1990).  NAEP.  On  the  cutting  edge  of  measurement  since  1969.  Princeton,  NJ. 
Educational  Testing  Service,  National  Assessment  of  Educational  Progress. 

Frankel,  M.  R.  (1971).  Inference  from  survey  samples.  Ann  Arbor.  Institute  for  Social  Research,  The 
University  of  Michigan. 

Hamblcton,  R.  K.  (1989).  Principles  and  selected  applications  of  item  response  theory.  In  R.  L.  Linn  (Ed.), 
Educational  measurement  (3rd  edition).  New  York.  American  Cou^  ^1  on  Education/Macmillan. 

Hammack,  D.  C,  Hartoonian,  M.,  Howe,  J.,  Jenki^is,  L.  B.,  J^vstik,  L.  S.,  MacDonald,      B.,  MullJs,  I.  V.  S., 
&  Owen,  E.  (1990).  The  U.S.  history  report  card:  The  achievement  of  fourth-,  eighth-,  and  twelfth- 
grade  students  in  1988  and  trends  from  1986  to  1988  in  the  factual  kno\^ledge  of  high  school  juniors. 
Princeton,  NJ:  Educational  Testing  Service,  National  Assessment  of  Ed'"*^»tional  Progress. 

Johnson,  E.  G.  ^1988).  Mathematics  data  analysis.  In  A.  E.  Beaton,  Expanding  the  nen  design.  The  WaEP 
1985-86  teamical  report  (pp.  215-240).  (No.  17-TR-20)  Princeton,  NJ.  Educational  Testing  Service, 
National  Assessment  of  Educational  Progress. 

Johnson,  E.  G.  (1989).  Considerations  and  techniques  for  the  analysis  of  NAEP  data.  JoumalofEd.  ^tional 
Statistics,  14,  303-334. 

Johnson,  Z.  G.  &  King,  B.  F.  (1987).  Generalized  variance  functions  for  a  complex  sample  survey.  Journal 
of  Official  Statistics,  3,  235-250. 

Johnson,  J.  R.  (1987).  Instrument  and  item  information.  In  A.  E.  Bealon,  Implementing  the  new  design 
The  NAEP  1983-84  technical  report  {pp.  U9-li4).  (No.  15-TR-20)  Princeton,  NJ:  Educational 
Testing  Service,  National  Assessment  of  Educational  rrogress. 

King,  B.  F.,  Bertrand,  R.  &  Dupuis,  F.  A.  (1989).  A  woM  of  differences.  An  intemational  assessment  of 
mathematics  and  science.  Technical  report.  Princeton,  NJ:  Educational  Testing  Service, 
International  Assessment  of  Educational  Progress. 


718 


Kirsch,  1.  S.,  &  Jimgeblut,  A.  (1986).  Literacy:  Profiles  of  America's  young  adults,  final  report,  (No. 
16-PL-Ol).  Princeton,  NJ;  Educational  Testing  Service,  National  Assessment  of  Educational 
Progress. 

Kish,  L.  (1965).  Survc*  sampling.  New  York:  John  Wiley  &  Sons. 

Kish,  L.  &  Frankel,  M.  R.  (1974).  Inference  from  complex  samples.  Journal  of  the  Royal  Statistix^al  5cH:<cr>, 
Series  S,  56, 1-22. 

Krewski,  D.  &  Rao,  J.  N.  K.  (1981).  Inference  from  stratified  samples;  Properties  of  linearization,  jackknlfL 
and  balanced  repeated  replication.  Annals  of  Statistics,  9, 1010-1019. 

Langer,  J.  A.,  Applebee,  A.  N.,  Mollis,  I.  V.  S.,  &  Foertsch,  M.  A.  (1990).  Learning  to  read  in  our  nation's 
schools:  Instruction  and  achievement  in  1988  at  grades  4,  8^  and  12.  Princeton,  NJ.  Educational 
Testing  Service,  National  Assessment  of  Educational  Progress. 

Little,  R.  J.  A.,  &  Rubin,  D.  B.  (1983).  On  jointly  estimating  parameters  and  missing  data.  America . 
Statisticiany  37,  218-220. 

Lord,  F.  M.  (1980).  Applications  of  item  response  ..^eory  to  practical  testing  problems.  Hillsdale,  NJ. 
Lawrence  Erlbaum  Associates. 

Messick,  S.  J.,  Beaton,  A.  E.,  &  Lord,  F.  M.  (1983).  NAEP  reconsidered:  A  new  desigti  for  a  new  era, 
(N\EP  Report  «^1)  Princeton,  NJ:  Educational  Testing  Service. 

Mislevy,  R.  J.  (1984).  Estimating  latent  distributions.  Psychometrika,  49{d),  359-381. 

Mislevy,  R.  J.  (1985).  Estimation  of  latent  group  effects.  Journal  of  the  American  Statistical  AssotLtion,  SO, 
993-997. 

Mislevy,  R.  J.  (1988a).  Randomization-based  inferences  about  latent  variables  from  complex  samplcji.  (ETS 
Research  Report  RR-88-54-ONR)  Princeton,  NJ:  Educational  Testing  Service. 

Mislevy,  R.  J.  (1988b).  Scaling  procedures.  In  A.  E.  Beaton,  Expanding  the  new  design.  Vie  NAEP  1985  86 
technical  report  {pp.  ITJ'TM).  (No.  17'TR-20)  Princeton,  NJ:  Educational  Testing  Service,  National 
Assessment  of  Educational  Progress. 

Mislevy,  R.  J.  (In  press).  Randomlzatlm-based  inferences  about  latent  variables  from  complex  sampler. 
Psychometrika. 

Mislevy,  R.  J.,  &  Bock,  R.  D.  (1982).  BILOG.  Item  analysis  and  test  scoring  with  binary  logistit  models 
[Computer  program].  Mooresville,  IN:  Scientific  Software. 

Mislevy,  R.  J.,  &  Bock,  R.  D.  (1989).  PGBILOG  3.  Item  analysis  and  test  scoring  with  bina.-y  lopstic 
models.  Mooresville,  IN:  Scientific  Software. 

Mislevy,  R.  J.,  &  Sheehan,  K.  M.  (1987).  Marginal  estimation  procedures.  In  A.  E.  Beat  n.  Implementing 
the  new  design:  The  NAEP  1983-84  technical  report  (pp.  293-360).  (No.  15'TR  20)  Princeton,  NJ. 
Educational  Testing  Service,  National  Assessment  of  Educcaonal  Progress, 

Mislevy,  R,  J.,  &  Sheehan,  K.  M.  (1989),  The  role  of  collateral  information  about  examinees  in  itLm 
parameter  estimation.  Psychometrika,  54,  661-679. 

Mislevy,  R,  J.,  &  Wu,  P-K.  (1988).  Inferring  examinee  ability  when  some  item  responses  are  missing,  (ETS 
Research  Report  RR-88-48-ONR)  Princeton,  NJ:  Educational  Testing  Service. 


ERLC 


719 


Mullis,  I.  V.  S.,  &  Jenkins,  L.  B.  (1990).  The  reading  report  card,  Trends  fron.  the  Nations  Report 

Card.  (No.  19-R-Ol)  Princeton,  NJ:  Educational  Testing  Service,  National  Assessment  of 
Educational  Progress. 

National  Assessment  of  Educational  Progress.  (1985).  Ihc  reading  report  card.  Progress  toward  excellence  m 
our  schools.  (No.  15-R-Ol)  Princeton,  NJ:  Educational  Testing  Service,  National  Assessment  of 
Educational  Progress. 

National  Assessment  of  Educational  Progress.  (1987a).  Reading  objectives,  1986  and  1988  assessments.  (No. 
17/19-R-lO)  Princeton,  NJ:  Educational  Testing  Service,  National  Assessment  of  Educational 
Progress. 

National  Assessment  of  Educational  Process.  (1987b).  Civics  objecdves,  1988  assessment.  (No.  19-C-lO) 
Princeton,  NJ:  Educational  Testing  Service,  National  Assessment  of  Educational  Progress. 

Rogers,  A.  M.  (1987).  Data  entry  system.  In  A.  E.  Beaton,  Implementing  the  new  design.  Thp  NAEP  1983- 
84  technical  report  {pp.  185-199).  (No.  15-TR-20)  Princeton,  NJ:  Educational  Testing  Service, 
National  Assessment  of  Educational  Progress. 

Rogers,  A.  M.,  Barone,  J.  L.,  &  Kline,  D.  L  (1989).  A  guide  to  the  NAEP  item  information  database. 
Princeton,  NJ:  Educational  Testing  Service,  National  Assessment  of  Educational  Progress. 

Rogers,  A.  M.,  Barone,  J.  L.,  &  Kline,  D.  L.  (1989).  A  guide  to  the  NAEP  restricted-use  data  files. 
Princeton,  NJ:  EducaLonal  Testing  Service,  National  Assessment  of  Educational  Progress. 

Rogers,  A.  M.,  Kline,  D.  L,  Barone,  J.  L.,  Mychajlowycz,  A.  W.,  &  Forer,  D.  C.  (1989).  A  primer  for  the 

NAEP  item  information  database.  Princeton,  NJ.  Educational  Testing  Service,  National  Assessment 
of  Educational  Progress. 

Rogers,  A.  M.,  Kline,  D.  L.,  Johnson,  E.  G.,  Mislevy,  R.  M.,  &  Rust,     F.  (1990).  National  Assessment  of 
Edi:cational  Progress  1988 public-use  data  tapes  version  2.0  user  guide.  Princeton,  VJ.  Educational 
Testing  Service,  National  Assessment  of  Educational  Progress. 

Rogers,  A.  M.,  Wine,  D.  L,  Johnson,  E.  G.,  &  Rust,  K  F.  (1989).  National  Assessment  of  Educational 
Progress  1988 public-use  data  tapes  version  1.0  user  guide.  Princeton,  NJ.  Educational  Testing 
Service,  National  Assessment  of  Educationa?  Progress. 

Rubin,  P.  B.  (1987).  Multiple  imputation  for  nonresponse  in  surveys.  New  York.  John  Wiley  &  Sons. 

Rust,  K  (1985).  Variance  estimatii  n  for  complex  estimators  in  sample  surveys.  Journal  of  Official  Statistits, 
1(4),  381-397. 

Rust,  K,  P.,  Bethel,  J^  Burke,     &  Ha^^n,  M.  H.  (1990).  The  1988  National  Assessment  of  Educational 
ProffessSampling  and  wei^ting  procedures,  final  report.  Rockville,  MD:  Westat,  Inc. 

Satterthwaite,  F.  E.  (1941).  Synthesis  of  variance.  Psychometrika,  6,  309-316. 

Shechan,  K.  M.  (1985).  M  GROUP:  Estimation  of  group  effects  in  multivariate  models  [Computer  piugiam]. 
Princeton,  NJ:  Educational  Testing  Service. 

S    :han,  K.  M.  (1988).  ITie  IRT  linking  procedure  used  to  place  the  1986  intermediary  scaling  results  onto 
the  1984  reading  calibration  scale.  In  A.  E.  Beaton,  Expanding  the  new  desipt.  The  NAEP  1985-86 
technical  report  (pp.  555-565).  (No.  17-TR  20)  Princeton,  NJ:  Educational  Testing  Service,  National 
Assessment  of  Educational  Progress. 


Stocking,  M.  L,  (1986).  TBLT:  Transforming  b*s  using  a  least  squares  technique  [Computer  program]. 
Princeton,  NJ:  Educational  Testing  Service; 

Mocking,  M.     &  Lord,  F.  M.  (1983).  Developing  a  ccmmon  metric  in  item  response  theory.  Applied 
Psychological  Measurementy  7,  201-210. 

Wilson,  D.,  Wood,  R.  L.,  &  Gibbons,  R.  D.  (1983).  TESTFACT:  Test  scoring  and  item  factor  analysis 
[Computer  program].  Chicago:  Scientific  Software. 

Wingersky,  M.,  Kaplan,  B.  A.,  &  Beaton,  A.  E.  (1987).  Jomt  estimation  procedures.  In  A.  E.  Beaton, 
Implementing  the  new  desifft:  The  NAEP  1983-84  technical  report  (pp.  285-292).  (No.  15-TR-20) 
Princeton,  NJ:  Educational  Testing  Service,  National  Assessment  of  Educational  Progress. 

Wolter,  K.  M.  (1985).  Introduction  to  variance  estimation.  New  York:  John  Wiley  &  Sons. 

Yamamoto,  K.  (1988).  Scsnce  data  analysis.  In  A.  E.  Beaton,  Ejqnmding  the  new  design:  The  NAEP  1985- 
86  technical  report  (pp.  243-255).  (No.  17-TR-20)  Princeton,  NJ:  Educational  Testing  Service, 
National  Assessment  of  Educ?tional  Progress. 

Yamamoto,  K.  (1990).  Mathematics  and  science  trend  data  analysis.  In  A.  E.  Beaton  &  R.  Zwick,  Tlie 
effect  of  changes  in  the  National  Assessment:  Disentangling  the  NAEP  1985-86  reading  anomaly  (pp. 
11M44).  Princeton,  NJ:  Educational  Testing  Service,  National  Assessment  of  Educational  Progress. 

Zwick,  R.  (1987a).  Assessing  the  dimensionality  of  NAEP  reading  data.  Joumal  of  Educational 
Measurement,  24, 293-308. 

Zwick,  R.  (1987b).  Use  of  the  NAEP  ahnanacs.  In  A.  E.  Beaton,  Implementing  the  new  design.  The  NAEP 
1983-84  technical  report  (pp.  517-522).  (No.  15-TR-20)  Princeton,  NJ:  Educational  Testing  Service, 
National  Assessment  of  Educational  Progress. 

Zwick,  R.  (1988).  Professionally  scored  items.  Technical  memorandmn. 


721 

ERIC 


INDEX 


723 


Focusing  the  New  Design:  The  NAEP  1988  Technical  Report 


INDEX 


Absentee  data 

adjustment  of,  for  makeup  sessions,  164 

data  entry  lor,  114 

editing  of,  155 
Administration  Schedule,  93,  95,  96 

materials  processing,  112>  114-15 
Age  definition,  15,  16 

for  all  student  samples,  16 

clianges  in,  7, 15 

for  civics  cross-sectional  samples,  299 

for  civics  trend  samples,  299 

effect  of  changes  in,  189 

for  geography  cross-sectional  samples,  326 

for  mathematics  trend  samples,  330,  332 

for  reading  cross-sectiocal  samples,  253 

for  reading  trend  samples,  253 

for  science  trend  samples,  330,  332 

for  U.S.  history  cross-sectional  samples,  318 

for  U.S.  history  trend  sample,  318 

for  writing  cross-sectional  smples,  268 

for  writing  trend  samples,  268 
Anchoring,  6,  182-83 

of  civics  cross-sectional  scale,  308-9 

of  reading  trend  scale,  256 

of  U.S  history  cross-sectional  scale,  322,  323 
Answer  documents.  See  also  Resp  '^nse 
methods/modes 

evolution  of,  101 

key-entered  booklets,  101 

and  placement  of  open-ended  item  scores,  128 

scannable  answer  sheets,  101,  110 

.scannable  booklets,  101 
ARM.  See  Average  response  method 
Assessment  instruments,  27-31,  69-83.  See  also 
Booklets;  Item  blocks;  Excluded  student 
questionnaire;  School  characteristics  and 
policies  questionnaire;  Teacher  questionnaire 
Assessment  items,  summary  statistics  for,  344.  See 

aiso  Item  blocks;  Item  development 
Assessment  Policy  Committee,  33,  34,  43,  47 
Assessment  i^essions 

summary  statistics  for,  345 

types  of,  60-63 
Assistant  field  directors 

responsibilities,  85 
Average  response  method  (ARM),  275-78 

application  to  the  writing  cross-sectional  scale, 
291-93 


application  to  the  writing  trend  scale,  287-88 
composite,  275,  276,  277 
computation  of  plausible  values  for,  240-42 
estimable  correlations  for  writing 

cross-sectional  analysis,  293 
estimable  correlations  for  writing  tr  id  analysis, 

281 

estimation  of  cross-products  matrix 
directly  estimable  terms,  278-82 
imputation  of  inestimable  correlations,  279, 
283-^ 

terms  estimable  from  other  grades  or  years, 
279,  282-83 
and  scaling,  235 

use  of,  in  writing  data  analysis,  182 

Background  and  attitude  que,<ition  development, 
26-27,  33-50 
for  civics  and  U.S.  history,  45 
for  geography,  46 
for  reading,  39 
for  writing,  42 
Balar  ^.d  incomplete  block  (BIB)  spiral  design, 
*-4-15,  27-29.  See  also  Focused-BIB  spiral 
design 

introduction  of,  in  NAEP,  4 
Bias 

consideration  of,  in  secondary  analyses,  245-247 
review  of  item  text  for,  34,  39,  42 
BIB-spiral  design.  See  Balanced  incomplete  block 

spiral  design 
BILOG  computer  program,  181,  261,  233,  239, 

305,  321,  327,  337 
Blocks.  See  Item  blocks 
Booklets 

arrangement  of  blocks  in,  for  main  sample^,  70 

assignment  to  students,  29 

blocks  used,  for  bridge  booklets,  71,  78-^2,  for 
focused-BIB  booklets,  69,  72-74;  for 
intercorrelatiop  booklets,  69,  72-74;  for 
special  study  booklets,  70,  72-74 

civics,  298,  311,  312 

design  of,  for  bridge  samples,  29-30;  for 
focused-BIB  samples,  6,  27-28;  for 
intercorrelation  samples,  6,  28 

mathematics  trend,  330 

nur^iber  administered,  for  bridge  samples, 
78-80;  for  -nain  samples,  72-74 


ERIC 


725 

751 


Booklets  (continued) 

number  created  for  main  samples,  71 

response  methods,  for  bridge  samples,  78-SC;  for 
mam  samples,  72,  73,  74;  for  reading,  254 

science  trend,  330 

summary  statistics,  344 

used  for  all  samples,  16 
Bridge  samples,  17-18, 179,  189-90.  See  also  Trend 
samples 

age  deflnition  for,  16 

block  contents  for,  78-80 

booklets  used  for,  16,  71,  7&-82 

for  civics  trend  analysis,  311 

introduction  of,  in  NAEP,  i 

for  mathematics  trend  analysis,  329 

modal  grades  for,  16 

mode  of  administration  for,  16 

number  of  booklets  administered  in,  78-80 

for  reading  anomaly  analysis,  6,  255 

for  reading  trend  analysis,  252,  255 

response  methods  for  booklets  in,  78-80 

sample  sizes  for,  16 

for  science  trend  analysis,  329 

student  cohorts  assessed  for,  16 

time  of  testing  for,  16 

for  U.^  history  trend  analysis,  317 

for  writLig  trend  analysis,  268 

Calculator  items,  331 

Chief  State  School  Officers,  31,  87,  88 

Civics  assessment  development 

background  questions,  45 

items,  44 

objectives,  43 
Civics  CTOSS-scctional  data  analysis,  297-310 

anchoring,  308 

conditioning  effects,  306,  574-85 
conditioning  variables,  306,  529-36;  composite 

and  derived,  306,  309,  620-28^ 
dichotomization  of  open-ended  items,  310 
differential  item  functioning,  305 
dimensionality,  301,  304-5 
effect  of  block  position,  301,  303 
item  analysis,  301 
item  calibration,  306 
item  classification,  310 
item  parameters,  305, 654-57 
items  excluded  from  scaling,  305 
items  used,  298, 300 

KR-20  reliabilities  for  item  blocks,  301,  302 

not-reached  percents  for  item  blocks,  301,  302 

parameter  estimation,  305 

plausible  values,  305-6 

populations  assessed,  297 

proficiency  estimation,  306-7 

reporting  variables,  composite  and  derived,  309, 


699-702 
sample  sii^s,  300 
samples  used,  298 
scaling,  306-9 

transformation  of  plausible  values,  307 
Civics  cross-sectional  samples 
age  definition  for,  299 
booklets  used  for,  299 
modal  grades  for,  299 
sample  sizes  for,  299 
time  of  testing  for,  299 
Civics  data  analysis,  297-316.  See  also  Civics 

cross-sectional  data  analysis;  Civics  trend 

data  analysis 
Civics  trend  data  analysis,  311-316 
conditioning  effects,  314,  586-92 
conditioning  variables,  314,  537-44;  composite 

and  derived  conditioning  variables  315,  628 
dichotomization  of  open-ended  items,  316 
equating,  314 
item  analysis,  313 
item  calibration,  313 
item  parameters,  314,  658-61 
item  response  function,  313 
items  excluded  from  scaling,  311,  313 
items  used,  311,  312 
KR-20  reliabilities  for  item  blocks,  313 
not-reached  percents  for  item  blocks,  313 
parameter  estimation,  313 
plausible  values,  314 
populations  assessed,  311 
proficiency  estimation,  315 
reporting  variables,  composite  and  derived,  315. 

702 

sample  sizes,  312 
samples  used,  311 

transformation  of  plausible  values,  315 
Civics  trend  samples 

age  dcHnition  for,  299 

booklets  used  for,  299 

modal  grades  for,  299 

mode  of  administration  for,  311,  313 

sample  sizes  for,  299 

time  of  testing  for,  299 
Coefficient  of  variation 

definition,  347 
Common  background  questions 

composite  and  derived,  for  reporting,  687-93 

content  of,  47 

development  of,  47 

timing  of,  47 
Conditioning  effects 

for  dvics  cross-sectional  data,  306,  574-85 

for  civics  trend  data,  314,  586-92 

for  geography  cross-sectional  data,  327,  599-600 

for  mathematics  trend  data,  334,  335,  601-6 
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for  reading  cross-seclional  data,  262,  559-68 
for  reading  teacher  data,  264,  569-70 
for  reading  trend  data,  256, 571-73 
for  sdencc  trend  data,  339,  607-9 
for  U.S.  history  cross-sectional  data,  322,  593-98 
Conditioning  variables 
for  1988  scales,  248,  249 
for  cmcs  cross-sectional  data,  306,  529-36; 

composite  and  derived,  306,  309,  620-21 
for  civics  trend  data,  31 537-44;  composite  anu 

derived,  315, 628 
common,  503-7 

for  geography  cross-seclional  data,  327,  549; 

composite  and  derived,  635-36 
for  mathematics  trend  data,  334,  335,  550-54; 

composite  and  derived,  637-38 
and  plausible  values,  238,  240 
for  reading  cross-sectional  data,  262,  508-19; 

composite  anii  derived,  613-14 
for  reading  teacher  data,  264 
for  reading  trend  data,  256,  520-22 
and  reduction  of  potential  bias  in  secondary 

analyses,  246 
for  science  trend  data,  339,  555 
for  U.S.  history  cross-sectional  data,  322,  545-48; 

composite  and  derived,  628-35 
for  writing  cross-sectional  data,  291,  523-25; 

composite  and  derived,  294-295,  615-18 
for  writing  trend  data,  275,  279,  526-28; 

composite  and  derived,  619-20 
Consultants 

for  objectives  and  item  development,  35,  471-80 

Data  analysis,  179-85.  See  also  Civics  data  analysis. 
Geography  data  analj'sis;  Mathematics  data 
analysis;  Reading  data  analysis;  Science  data 
analysis;  Teacher  data  analysis;  U.S.  history 
data  analysis;  Writing  data  analysis 

anchoring,  182-83 

average  response  method,  182 

dichotomization,  181 

diflbrential  stem  functioning,  181 

item  analysis,  180 

item  response  function,  181 

item  response  theory,  181 

parameter  estimation,  181 

plausible  values,  181 

proficiency  estimation,  181 

reliability  for  item  blocks,  180 

samples  of  students,  179-^ 

scaling,  181-83 

spcededncss  checks,  180 
Data  entry 

for  absentee  data,  114 

for  excluded  student  questionnaire,  123 

for  school  characteristics  and  policies 
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questionnaire,  124 
for  school  worksheet,  111-12 
system,  152-53 

for  teacher  questionnaire,  124 
Data  processing.  See  also  Data  transcription, 

Database  creation;  Editing  data;  Materials 

processing 
flow,  102 

scope  of  work,  100 

systems,  100 
Data  transcription,  145-53 

accuracy  of,  100 

entry  and  resolution,  152-53 

for  excluded  student  questionnaire,  102 

loading,  150-52 

quantities,  100 

scanning,  145,  148-49 

for  school  questionnaire,  102 

for  teacher  questionnaire,  102 

time  period  for,  100 
Database.  See  also  Database  creation 

overview  of,  105 

size,  107 

structure,  106 
Database  creation,  105, 163-66 

derivation  of  weights  for,  163-64 

master  catalog  for,  165-66 

mer^ng  files  for,  164-65 
Database  products,  167-75  See  also  Item 
information  database;  Public-use  data 
tapes/files;  Restricted-use  data  files 

usefulness,  8, 106 
Degrees  of  freedom 

of  jackknife  variance  estimator,  220,  222-28 
Design  and  Analysis  Cororclttee,  members  of,  4 
Design  effects 

for  proportion-correct  statiitics,  212-17 

for  simple  regression  coefficients,  220,  221 

for  subgroup  mean  proficiency  scores,  217-19 

use  of,  in  estimating  sampling  variability, 
212-20 

Design  of  assessment,  13-31.  See  also  Assessment 
instruments;  Background  and  attitude 
question  development;  Item  development; 
Objectives  development;  Sample  design 

a^'e  definition,  15 

assessment  instruments,  27-?^l 

background  question  development,  26-27 

BIB  spiral  design,  14-15,  27-29 

booklet  design,  27-30 

bridge  samples,  17-18 

considerations  in,  14 

focused-BIB  samples,  15,  17 

focused-BIB  spiral  design,  15,  28 

half*samples,  17 

improvements  in,  14 
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Design  of  assessment  (condnued) 

intercorrelation  samples,  17 

item  block  design,  27 

item  development,  26-27 

modal  grades,  15 

objectives  development,  26-27 

overview  of  de«gn,  14-15 

populations  assessed  smcc  11^9,  1^22 

sample  selection,  19,  23-26 

spcdal  study  samples,  17 

subject  area^J  assessed  m  1988,  13;  since  1969, 
19-22 

time  of  testing,  15 
Dichotomization  of  open-ended  items,  181 

for  dvacs,  310,  316 

for  reading,  257 

for  JJS.  history,  319 
Diflcrential  item  hinctioning  (DIF),  181 

for  civics  cross-sectional  data,  305 

consideration  of,  m  scaling,  233 

for  reading  cross-sectional  data,  261 
Dimensionality 

of  civics  aoss-scctional  items,  301,  304-5 

consideration  of,  m  scaling,  134 

of  read^  3  trend  items,  255 
District  supcrwors 

nuajbcrs  of,  86 

rcsponsibilit!-^  86,  90, 91, 93-94 
training,  86-87 
Document  literacy,  7,  180 
booklet  coDteata;  for,  70 
number  of  open-ended  items  for,  127 
preliminary  item  analysis  for,  2S2 
scoring  of  open-ended  items  for,  131 

Editing  data,  155-58 

for  absentees,  155 

for  assessed  students,  156 

for  professionally  scored  items,  157 

for  questionnaires,  156-57 
Education  Coi;:mission  of  the  Stales,  229,  242 
Eligibility 

distribution  of  students  by,  345 

for  students  m  bridge  samples,  23 

for  students  m  mam  samples,  19 
Equatmg 

for  civics  trend  data,  314 

for  mathematics  trend  data,  330,  331,  334,  335 

for  reading  cross-sectional  data,  263 

and  scaling,  235 

for  science  trend  data,  330,  331,  337 
Error  analysis.  See  Quality  control 
"Estimate*  items,  335 
Excluded  student  questionnaire 

content,  47,  48,  83 

data  transcription,  102 


development,  47 

editing  data  from,  156-57 

and  field  administration,  95,  96 

materials  processing  for,  122-25 

number  of  items  in,  344 

quality  control  error  analysis  for,  161 
Excluded  students 

distribution  o!^  by  age  dias  and  sample,  345 

mtroduction  of,  in  NAEP,  4 

population  estimates  (weighted)  for,  347 

rates  of  exclusion,  66, 96,  98 

and  sample  design,  64,  66 
Exercise  admbistratr-fs,  93,  94 

recruitment,  86,  91 

responsibilities,  86,  93,  95^  96 

training,  91 

Field  administration,  31-32,  85-98.  See  also  Field 
administration  foims;  Field  administration 
staff 

arranpng  assessments,  94 

distributing  and  collecting  questionnaires,  95-96 

distiict  supervisors'  responsibilities,  86,  90,  91, 

93-94 
makeup  sessions,  93 

obtaining  cooperation  of  districts  and  schools, 

87-91 
organization  of,  85-S6 
participation  rates,  for  schools,  90,  92;  for 

students  in  fall  assessments,  96,  97;  for 

students  in  winter/cpring  assessments,  97,  98 
preparbg  reports  and  shipping  materials,  96 
sample  sizes,  for  students  in  fall  assessments, 

96,  97;  for  students  m  winter/spring 

assessments,  97,  98 
selection  of  students,  94-95 
training  of  dii,^.rict  supervisors,  86-87 
Field  administration  forms.  See  Ad*ninistration 

Schedule;  Introductory  Meeting  Form; 

Results  of  Cop.taa  Form;  Roster  of 

Questionnaires;  School  Update  Foim,  School 

V/orksheet;  Session  Assignment  Form; 

Student  Listiif-g  Form;  Teacher  Su.^ey  Roster 
Field  adm?nistrati()u  staff.  See  Assistant  fiela 

directors,  DLtrict  supervisors;  Exercise 

adminislraiofs;  Field  director;  Project 

director;  School  coordinators 
Field  director,  94 

responsibilities,  85-87 
Feld  testing 

and  item  development  process,  48-49 
Focused-BIB  samples,      17,  180 
age  definition  for,  16 
block  used,  72-74 
booklet  contents,  69 
booklets  used  for,  16 
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lOr  dvics  cross-sectional  analysis,  297,  298 

for  geography  analysis,  325 

modal  grades  for,  16 

mods  of  administration  for,  16 

number  of  booklets  administered  for,  72-74 

for  reading  cross-sectional  analysis,  252,  256 

response  methods  used  for,  72^-74 

sample  sizes  for,  16 

student  cohorts  assessed  for,  16 

time  of  testing  for,  16 

for  U.S.  history  cross-sectional  analysis,  317 
for  writmg  aoss-sectional  analysis,  268,  289 
Focused-BIB  spiral  design,  5-^,  15, 28 
and  reduction  of  potential  bias  in  secondary 
analyses,  246 

Geography  assessment  development 

background  questions,  46 

items,  46 

objectives,  45 
Geography  cross-sectional  data  analysis,  325-28 

conditioning  effects,  327,  599-600 

conditioning  variables,  327,  549;  composite  and 
derived,  635-36 

item  analysis,  325 

item  calibration,  327 

item  classification,  327-28 

item  parameter  estimation,  327 

item  parameters,  666-67 

item  response  theory,  327 

items  excluded  from  scaling,  325 

KR-20  reliabilities  for  item  blocks,  326 

mean  proportion  correct  for  item  blocks,  326 

sot-  reached  perccnts  for  item  blocks,  325 

plausible  values,  327 

proficiency  estimation,  327 

reporting  variables,  composite  and  derived,  704-7 

samples  used,  325 

scaling,  327 
Geography  cro's-sectional  samples 

age  definition  for,  326 

booklets  used  for,  326 

modal  grades  for,  326 

sample  sizes  for,  326 

time  of  testbg  for,  326 
Geography  dala  analysis.  See  Geography  cross- 
sectional  data  analysis 
Grade/age.  See  Student  (X)horts 

Half-samples 

purpose  of,  17 

student  weights  for,  202 
Holistic  data 

analysis  of,  for  writing  trend,  288 

correlation  of  scores  with  primary  trait  scores. 
132 


items  used  for,  in  writing  trend,  289 
sample  sizes  for  writing  trend,  289 
scoring,  133-34 
training  of  scorers,  138-39 

Implementation  of  assessment.  See  Field 
administration;  Materials  processing; 
Professional  scoring 
Imputation.  See  Plausible  values 
Instruments,  27-31,  69-83.  See  also  Booklets;  Item 
blocks;  Excluded  student  questionnaire; 
School  characteristics  and  policies 
questionnaire;  Teacher  questionnaire 
Intercorrelation  samples,  17, 180 
age  definition  for,  16 
block  used  for,  72-74 
booklet  contents  for,  69 
booklets  used  for,  16 
for  civics  cross-sectional  analysis,  297 
for  geography  cross-seclional  analysis,  325 
modal  grades  for,  16 
mode  of  administration  for,  16 
number  of  booklets  administered  for,  72-''4 
purpose  of,  6 

for  reading  cross-sectional  analysis,  252, 256 
response  methods  used  for,  72-74 
sample  sizes,  16 

student  cohorts  assessed  for,  16 
time  of  testing  for,  16 

for  U.S.  history  cross-sectional  analysis,  317 
Intraclass  correlation  cocfiicient,  calculation  of, 
140n 

Introductory  Meeting  Form,  88,  89,  90 
IRT,      Item  response  theory 
Iter-  -tt'.-Jlysis,  180 

^r.  -.u^ration  of,  in  scaling  234 

hr  Jvics  cross-sectional  data  301 

for  dvics  trend  data  313 

for  document  literacy  items  252 

for  geography  cross-sectional  data  325 

for  mathematics  trend  data  331 

for  reading  cross-sectional  data  258 

for  reading  trend  data  255 

for  science  trend  data  337 

for  U.S.  iiistory  cross-sectional  data  319 
Item  blocks 

arrangement  of,  70 

assembly  of,  for  field  test,  39;  for  final 

assessment,  49 
background  items  :n,  for  main  samples,  75-77 
cognitive  blocks:  number  created  for  main 

samples,  70;  use  in  bridge  sample  booklets, 

78-80;  use  in  main  sample  booklets,  72-74 
cognitive  items  in,  for  main  samples,  75-77 
common  background,  69;  used  in  bridge  sample 

booklets,  78-80;  used  in  main  sample 
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Itcn!  blocks  (continued) 
booklets,  72-74 
design  of,  27 

effect  of  position  of,  in  booklets:  for  dvics,  301s 
303;  for  reading,  258;  for  writing,  294 

for  mathematics  trend  samples  330 

for  science  trend  samples  330 

subject  orca  background:  use  m  bridge  sample 
booklets,  78-SO;  us^  in  main  sample  booklets, 
72-74 

timing  of,  for  bridge  samples,  71,  81,  82;  for  main 
samples,  70 

total  of,  by  subject  area,  50 

total  open-ended  items  in,  for  main  samples, 
75-77 
Item  calibration 

for  civics  cross-sectional  data,  306 

for  dvics  trend  data,  313 

for  geography  cross-sectional  data,  327 

for  reading  cross-sectional  data,  261 

for  reading  trend  data,  256 

for  U^.  history  cross-sectional  data,  321 
Item  dassiflcation 

for  dvics  cross-sectional  data,  310 

for  geography  cross-sectional  data,  327-28 

'  .  U.S.  hktory  cross-sectional  data,  323-24 
Item  development,  26-27,  33-50 

for  dvics  and  U.S.  history,  44 

consensus  process,  34 

consultants  for,  35,  471-80 

general  procedures  for,  37-38 

for  geography,  46 

major  considerations  in,  34 

for  reading,  39 

sdiedule,  35 

for  writing,  41 
Item  information  database,  8,  99, 167-68 

CTeation  procedures,  167 

overview,  106 

purpose  of,  167 

size,  100 

structure,  167 
Item  response  function,  181 

for  dvics  trend  data,  3D 

for  mathematics  trend  data,  335 

for  reading  cross-sectional  data,  261 

for  reading  trend  data,  256 
Item  response  theory  (IRT).  See  also  Item  response 
theory  parameter  estimation;  Item  response 
theory  parameters 

introduction  of,  in  NAEP,  4 

and  scaling,  232 

use  in  data  analysis,  181 
Item  response  theory  parameters 

for  dvics  cross-sectional  data,  305,  654-57 

for  dvics  trend  data,  314,  658-61 


for  geography  cross-sectional  data,  666-67 

for  mathematics  trend  data,  334,  335,  668-72 

for  reading  cross-sectional  data,  262,  647-50 

for  reading  trend  data,  256, 651-53 

for  sdence  trend  data,  337, 673-77 

for  U.S.  history  cross-sectional  data,  662-65 

Item  response  theory  parameter  estimation,  181 
for  d^cs  CTOSS-sectional  data,  305 
for  dvics  trend  data,  313 
for  geography  cross-sectional  data,  327 
for  mathematics  trend  data,  331,  335 
for  reading  cross-sectional  data,  261 
for  reading  trend  data,  256 
and  scaling,  233 
for  sdence  trend  data,  337 
for  U.S.  history  cross-sectional  data,  321 

Items  used  m  assessment,  summary  statistics  for, 
344.  See  also  Item  blocks;  Ite.n  development 

Items  exdudcd  from  -caling 
for  dvics  cross-sectional  analysis,  305 
for  dvics  trend  analysis,  311,  313 
lor  geography  cross-sectional  analysis,  325 
for  mathematics  trend  analysis,  331 
for  reading  trend  analysis,  256 
for  sdence  trend  analysis,  331 
for  U.S.  history  cross-sectional  analysis,  321 

Jackknife  variance  estimator.  See  Sampling 
variability,  estimation  of 

Key  entry 

quantities,  100 
KR-20  reliabilities 

for  dvics  cross-sectional  item  blocks,  301,  302 

for  dvics  trend  item  blocks,  313 

for  geography  cross-sectional  item  blocks,  326 

for  reading  cross-sectional  item  blocks,  258 

for  reading  trend  item  blocks,  255 

for  U.S.  history  cross-sectional  item  blocks,  319, 
320 

Learning  Area  Committees 

and  item  development,  34,  35,  37,  39-42,  44-46 
and  objectives  development,  34,  35,  36,  40, 

43-45 
selection  of,  35,  36 

M-GROUP  computer  program,  181,  238,  305,  327, 
334 

Main  samples,  15-17, 189.  See  also  Focuscd-BIB 
samples;  Intercorrelation  samples;  Special 
study  samples 
Makeup  sessions,  93 

number  of,  by  age  class  and  sample,  345 
Materials  processing,  102, 109-25 

for  administration  schedules,  112,  114-15 
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data  entry,  111,  113, 114,  123, 124 
flow,  102 

loading  scanned  data,  117-19, 123 

for  professionally  scored  respci?ses,  115-16 

quality  control,  120, 125 

for  question  naires>  122 

receipt  of  materials,  109-11 

resolution  processing,  119-22, 123 

scanning,  110,  116-17, 123 

for  school  worksheets,  111-12, 113 

for  student  instruments,  115-22 

validation,  120, 124 

verification,  1^ 
Mathematics  data  analysis.  See  Mathematics  trend 

data  analysis 
Mathematics  trend  data  analysis,  329-337 

conditioning  effects,  334,  335,  601-6 

conditioning  variables,  334,  335,  550-54; 
compo;ite  and  derived,  637-38 

equating,  330,  331,  334-35 

"estimate"  items,  335 

item  analysis,  331 

item  parameter  estimation,  331,  335 
item  parameters,  334, 335,  668-72 
item  response  function,  335 
items  excluded  from  scaling,  331 
m;un  objective  of,  330 

mean  proportion  correct  for  item  blocks,  333 

plausible  values,  334 

profidency  estimation,  334 

proficiency  scores,  335 

scaling,  331^  »3-37 
Mathematics  trend  samples 

agf'  definition  for,  330,  332 

booklets  for,  330 

modal  grades  for,  332 

mode  of  admbistration  for,  329,  330 

sample  sizes,  332 

time  f  f  testing  for,  330,  332 
Mechanics  data 

analysis  of,  for  writing  treni^  288 

items  used  for  writing  trend,  288 

sample  sizes  for  writmg  trend,  2?8 

scoring,  133 

training  of  scorers  for,  138 
Modal  grades 

for  all  student  samples,  16 
changes  in^  15 

for  civics  cross-sectional  samples,  299 

for  civics  trend  samples,  299 

for  geography  cross-sectional  samples,  326 

introduction  of,  in  NAEP,  5 

for  mathematics  trend  samples,  332 

for  reading  cross-sectional  samples,  253 

for  reading  trend  samples,  253 

for  science  trend  samples,  332 
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for  U.S.  history  cross-sectional  samples,  318 
for  U.S.  history  trend  sample,  318 
for  writing  cross-sectional  samples,  268 
for  writing  trend  samples,  268 
Mode  of  administration 
for  all  student  samples,  16 
Tor  dvics  trend  samples,  311,  313 
effect  of  changes  in,  189 
for  mathematics  trend  samples,  329,  330,  332 
for  science  trend  samples,  329,  330,  332 

National  Center  for  Education  Statistics,  4 
National  Geographic  Society,  33,  34,  45 
Nomesponse,  adjustments  for.  See  Weighting 

procedures 
Not-reached  percents 

for  civics  cross-sectional  item  blocks,  301,  302 

for  civics  trend  item  blocks,  313 

conaderation  of,  in  scaling,  234 

for  geography  cross-sectional  item  blocks,  325 

for  reading  cross-sectional  item  blocks,  258,  259 

for  reading  trend  items,  255 

for  U.S.  history  cross-seclional  item  blocks,  319 

Objectives  booklets 

review  and  publication,  49 
Objectives  development,  26-27,  33-50 
for  dvics,  43 
consensus  process,  34 
consultants  for,  35,  471-80 
general  procedures  for,  35-36 
for  geography,  45 
for  reading,  38 
major  considerations  in,  34 
schedule,  35 
for  U.S.  history,  43 
for  writing,  40 
Office  of  Educational  Research  and  Improvement 

(OERI)  review  of  publicationc,  35 
Office  of  Management  and  Budget  (0MB) 

dearance 
for  all  iter»:s,  35,  49,  50 
for  background  questionnaires,  47 
for  dvics  and  U.S.  history  items,  44 
for  geography  items,  46 
for  reading  items,  39 
for  writing  items,  42 
Open-ended  items 
and  dvics  data  analysis,  298,  300,  310  316 
numbers  of,  in  bridge  s<  aples,  127;  in  main 

samples,  127 
and  reading  data  analysis,  256,  257,  253,  263 
response  times  for,  in  bridge  samples,  130;  in 

main  samples,  129 
score  ranges  for,  in  bridge  samples,  130;  in 

main  samples,  129 
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Open-ended  items  (continued) 
scorer  reliability  for,  in  bridge  samples,  145-44; 

in  main  samples,  141-42 
total  of,  in  each  main  sample  block,  75-77 
and  U.S.  history  data  analysis,  317,  319 
Oversampling 
of  Black  and  Hispanic  students,  5,  52,  57, 188, 
190-191 

of  Black  students,  for  writing  aechv^'ics  aualysis, 
288 

Paced  tape.  See  Mode  of  admimstralion 
Parental  notification  or  permission,  94 
Participation  rates 
for  schools,  in  bridge  samples,  61,  90,  92;  in  main 

samples,  60,  90,  92;  summary,  90,  92 
for  students,  in  fall  assessments,  96,  97;  in 

winter /spring  assessments,  97,  98;  by  age  class, 
66 

Percent  correct 
definition,  258^ 

for  dwcs  cross-sectional  item  blocks,  302 
for  d^cs  trend  item  blocks,  3U 
for  reading  cross-sectional  item  blocks,  260 
Plausible  values,  236-248 
for  dwcs  cross-sectional  scale,  305-6 
for  dwcs  trend  scale,  314 
computation  of,  in  average  response  method 
scales,  240-42;  in  IRT-based  scales,  237-40 
and  conditioning  variables,  238,  240 
and  data  analysis,  181 
on  database,  165 

example  of  use  in  secondary  analyses,  247-48 

as  intermediary  computations  only,  237 

for  mathematics  trend  samples,  334 

methodology-,  for  scaling,  235-37 

for  noorespon  dents,  237 

oa  public-use  data  tapes,  243 

purpose  of,  6,  236 

teacher-based,  for  readi**";,  264 

transformation  to  profi  .cncy  scale,  182;  for  dvics 
cross-sectional,  307;  for  dwcs  trend,  315;  for 
geography  aoss-sectlonal,  327;  for 
mathematics  trend,  334,  337;  for  sdence  trend, 
337,  339;  for  U.S.  history  cross-sectional,  322 

for  U.S.  history  cross-sectional  scale,  322 

use  of,  in  secondary  analyses,  242-48 

for  writing  cross-sectional  scale,  291 

for  writing  trend  scale,  275-278,  288 
Population  estimates 

exduded  stu<Ients,  347 

students,  346 
Populations  assessed 

in  1988, 188 

since  1969, 19-22 
Poststratification,  198-202 


to  adjust  for  oversampling  in  writing  trend 

analysis,  288 
changes  in  procedures,  199-201 
effect  of  changes  in  procedures,  201-202, 

203-205 

for  exduded  student  weights,  ''06 
improvements  in,  8 

revisions  for  1984  weights,  201,  641-42 
for  teacher-student  we'ghts,  207 
Primary  samplmg  units  (PSUs),  23-24,  52-56 
certainty  and  noncertainty,  53 
designation  by  seas'^n  for  main  assessments,  55 
designation  for  bridge  assessments,  55-56 
drawn  for  multiple  assessment  years,  55 
final  sample  of,  54 
formation  of,  52-54 

stratification  of,  by  geographic  region,  53;  by 

minority  population,  54;  by  MSA  or 

nonMSA,  53;  by  socioeconomic 

characteristics,  54 
Primary  trait  data 

agreement  and  reliability  of  scores,  139-144, 

271-273 

correlation  of  scores  with  hol-otic  scores,  132 
effect  of  across-year  variation  •  i  scores,  270 
analysis  of,  for  writing  cross-sectional,  289-91; 

for  writing  trend,  268,  270-75 
sample  sizes,  for  writing  cross-sectional,  290, 

for  writing  trend,  274 
scoring,  132-33 
training  for  scoring,  137-38 
Prindpal  questionnaire,  60 
Professional  scoring,  127-44.  See  also  Open  cnded 
item.v.  Primary  trait  data 
answer  documents  and  placement  of  scores,  128 
editing  data  for,  157 

guidelines  for  scoring  of  open-ended  items,  for 
d^dcs,  135-36;  for  document  literacy,  131;  for 
mathematics,  134-35;  for  reading,  128, 131; 
for  sdence,  135;  for  U.S.  history,  136;  for 
writing,  131-34 

holistic  scoring,  133-34 

materials  processing,  115-16 

mechanics  scoring,  133 

number  of  items  scored,  100 

number  of  responses  scored,  100 

primary  trait  scoring,  132-33 

reliability  and  resolution,  139-44 

scoring  supervisor,  137 

scoring  teams,  128, 137 

training  of  scorers,  137-39 

work  schedule,  139 
Proficiency  estimates 

of  students,  by  subpopulation,  347 
Profidency  estimation,  181 

difference  in,  for  reading  trend,  255 
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for  civics  crosr  sectional  scale,  306-S 

for  civics  trend  scale,  315 

for  geography  cross-sectional  scale,  327 

for  mathematics  trend  samples,  334,  335 

for  reading  cross-sectional  scale,  262 

for  reading  trend  scale,  256 

for  science  trenc!  samples,  337 

for  U.S.  history  cross-sectional  scale,  322 
Project  director  (field  administration) 

responsibilities,  86,  87 
Proportion  correct 

for  geography  cross-sectional  item  blocks,  3*26 

for  mathematics  trend  item  blocks,  333 

for  science  trend  item  blocks,  338 

for  U.S.  history  cross-sectional  item  blocks,  320 
F^»7s.  See  Primary  sampling  units 
Public-use  data  tapes/files,  8,  99,  t68-.5 

contents,  14 

codebooks  for,  173-74 

control  statement  files  for,  174-75 

data  definition  for,  171-72 

data  file  catalogs  for,  173 

data  file  layouts  for,  172-73 

file  definition  for,  169-70 

machine-readable  catalog  files  for,  175 

overview,  106 

plausible  values  on,  243 

purpose  of,  168,  343 

size,  100 

variables  definition  for,  170-71 

Quality  control 

confidence  limits  for,  159,  161 
of  data  entry,  159-62 

error  analvsis,  for  questionnaire  data  entry,  161; 

for  student  data  entry,  161 
for  materials  processing,  120, 125 
observed  error  rate  for,  159, 160 
quantities  involved  in,  100 
Quality  Education  Data,  Inc.  (QED),  57,  59, 165 
Questionnaires.  See  Excluded  student  questionnaire; 

Principal  questionnaire;  School  characteristics 

and  policies  questionnaire;  Teacher 

questionnaire 

Reading  assessment  development 

background  questions,  39 

items,  39 

objectives,  38 
Reading  anc:naly,  179,  235 

data  analysis  for,  251 

samples  used  for  analysis  of,  6,  255 
Reading  cross-sectional  data  analysis,  256-65 

conditioning  effects  262,  559-68;  for  teacher 
data,  264,  569-570 

conditioning  variables  262,  508-19;  for  teacher 
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data,  264;  composite  and  derived,  613-14 
dichotomization  of  open-ended  items,  257 
differential  item  functioning,  261 
effect  of  block  position,  258 
equating,  263 
item  analysis,  258 
item  calibration,  261 
item  parameters,  262,  647-50 
item  response  function,  261 
KR-20  reliabilities,  258 

not~reached  percents  tor  xXqxax  Mocks,  258,  259 

number  of  items  administered,  257 

parameter  estimation,  261 

percent  correct  for  item  blocks,  258,  260 

proficiency  estimation,  262 

purpose  of,  257 

samples  used,  252,  256 

scaling,  262 

scaling  metric,  262-64 

speedednes.s,  258 

teacher  data  analyses,  264-65 

teacher-based  plausible  values,  264 
Reading  cross-sectional  samples 

age  definition  for,  253 

booklets  used  for,  253 

modal  grades  for,  253 

response  modes  for,  254 

sample  sizes,  253 

time  of  testing  for,  253 
Reading  data  analysis,  251-65.  See  also  Reading 
cross-sectional  data  analysis;  Reading  trend 
data  analysis 

and  reading  anomaly,  251,  255 

document  literacy  items,  252 

main  goals  of,  251 

sample  characteristics,  253 
Reading  trend  data  analysis,  255-56 

anchoring,  256 

conditioning  effects,  256,  571-73 
conditioning  variables,  256,  520-22 
dimensionality,  255 
item  analysis,  255 
item  calibration,  256 
item  parameters,  256,  651-53 
item  response  function,  256 
items  excluded  from  scaling,  256 
KR-20  reliabilities  for  item  blocks,  255 
not-reached  percents  for  items,  255 
parameter  estimation,  256 
proficiency  estimation,  256 
purpose,  255 

reporting  variables,  comoosite  and  derived, 

695-96 
samples  used,  252,  255 
scaling,  256 
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Reading  trend  samples 

age  definition  for,  253 

booklets  used  for,  253 

modal  grades  for^  253 

response  modes  for,  254 

sample  sf^es,  253 

time  of  testing  for,  253 
Reliability 

internal  consistency,  for  item  blocks,  180;  for 
dvics  cross-sectional  blocks,  301,  302;  for  dvics 
trend  blocks,  313;  for  geography  cross-sectional 
blocks,  325;  for  reading  cross-sectioaal  blocks, 
258;  for  reading  trend  blocks,  255;  for  U.S. 
history  cross-sectional  blocks,  319 

of  professional  scoring  results,  139-44 

rater,  for  writing  trend  scores,  270-273 
Reporting  subgroups,  681-86 

age,  686 

gender,  681 

grade  in  school,  685-86 

parental  education,  684 

race/ethnidty,  imputed,  681-82 

race/ethnidty,  observed,  681 

region  of  the  country,  684 

size  and  type  of  community  (STOC)j  682-83 
Reporting  variables,  composite  and  derived 

civics  CTOss-sectional,  309,  699-702 

dvics  trend,  315,  702 

common  background,  687-93 

geography  cross-sectional,  704-7 

reading  trend,  695-96 

U.S.  history  cross-sectional,  703-4 

writing  cross-sectional,  696-97 

writmg  trend,  697-99 
Reports 

list  of,  for  1988  subject  areas,  3n 
RESOLVE  computer  program,  263 
Response  iXiethods/modcs.  See  also  Answer 
documents 

for  main  sample  booklets,  72-74 

for  reading  booklets,  254 
Restricted-use  data  fdes,  8,  99,  168 

creation  procedures,  168 

overview,  106 

purpose  of,  168 

structure,  168 
Results  of  Contact  form,  88-90 
Roster  of  Questionnaires,  93 

Sample  design,  19,  23-26,  51-68,  188-190 
assignment  of  sessions  to  schools,  60-63 
exduded  students,  64,  66 
exdusion  rates  for  students,  66 
oversampling  of  mmority  students,  52,  57 
partidpation  rates  for  schools  in  bridge  samples, 
61;  in  main  samples,  60 


partidpation  rates  for  students,  66 

partidpation  rates,  overaLV  68 

populations  sampled,  51 

primary  sampling  units,  52-56 

sample  sizes  for  schools  in  bridge  samples,  61; 

m  main  samples,  60 
selection  of  schools  for  bridge  samples,  60-63; 

for  main  samples,  57-^ 
selection  of  students,  63-64 
selection  of  teachers,  67 
session  types  for  bridge  samples,  60 
size  and  type  of  community  (STOC),  60,  682-83 
stages  of  selection,  51 
time  of  testing,  52 
types  of  assessment,  52 
Sample  size(s) 

for  all  student  samples,  16 

for  dvics  cross-sectional  samples,  299,  300 

for  civics  trend  samples,  299,  312 

for  geography  cross-sectional  samples,  326 

for  mathematics  trend  samples,  332 

for  reading  cross-secticaal  samples,  253 

for  reading  trend  samples,  253 

for  schools  in  bridge  samples,  61;  in  main 

samples,  60;  by  session  type,  65 
for  science  trend  samples,  332 
for  students  in  fall  assessments,  96,  97;  in 

winter/spring  assessments,  97,  98;  by  session 

type,  65 

for  U.S.  history  cross-sectional  samples,  318 
for  U.S.  history  trend  sample,  318 
for  writing  cross-sectional  primary  trait  analysis, 
290 

for  writing  trend  holistic  analysis,  289 
for  writing  trend  mechanics  analysis,  288 
for  writing  trend  primary  trait  analysis,  274 
summary  statistics,  344-45 
Sample  weights.  See  also  Weighting  procedures 
and  database  creation,  163-64 
excluded  student  wei^ts,  206 
jackknife  replicate  weights,  207,  208,  210 
student  full-sample  wt:ght,  202,  206 
student  season-spedfic  weight,  202,  206 
teacher-student  weights,  206 
used  for  reading  teacher  analysis,  265 
Samples  of  students,  15-23, 179-80.  See  aLw 

Bridge  samples;  Focused-BIB  samples; 

Intercorrelation  samples;  Special  study 

samples 

Sampling  variability,  estimation  of,  187-88,  207-28 
degrees  of  freedom  of  jackknife  variance 

estimator,  220,  222-28 
effect  of  sample  design  on,  187,  208 
jackknife  replicate  weights,  208,  210 
using  design  effects,  212-20 
using  jackknife  variance  estimator,  208-11 
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Scaling,  181-83,  229-49.  See  also  Plausible  values 
advantages  and  disadvantages,  230 
average  response  method  (ARM)  model  for,  235 
biases  in  secondary  analyses,  245-47 
and  changes  in  item  context,  235 
and  changes  in  speededness  conditions,  235 
for  civics  cross-sectional  data,  306-9 
for  civnics  trend  data,  313-315 
comparison  of  reporting  scale  and  Consumer 

Price  Index,  230-31 
computing  plausible  values  in  ARM  scales, 

240-42 

computing  plausible  values  in  IRT-based  scales, 
237-40 

detection  of  conditional  dependence  in,  233 
and  differential  item  functioning,  233 
and  dimensionality,  234 
and  equating,  235 

eX/'imple  of  use  in  secondary  analyses,  247-48 

for  geography  cross-sectional  data,  327 

?jid  item  response  theory,  232 

and  item  analysis,  234 

for  mathematics  trend  data,  331,  333-37 

methodology,  231-42 

metric,  for  reading  cross-sectional  data,  262-64 

and  not-reached  items,  234 

overview  of  1988  scales,  24»-49 

parameter  estimation,  233 

plausible  values  methodology  for,  235-37 

for  reading  cross-sectional  data,  262 

for  reading  trend  data,  256 

for  science  trend  data,  337-39 

in  secondary  analyses,  242-48 

statistical  tests  for  secondary  analyses,  244-45 

three-parameter  logistic  IRT  model  for,  232-35 

for  U.S.  history  cross-sectional  data,  319,  321-22 

for  writing  cross-sectional  data,  291-93 

for  writing  trend  data,  275-88 
Scanning,  110,  116-17, 123,  145, 148-49 

quantities,  100 
School  characteristics  and  policies  questionnaire  60 

contents,  47,  48,  83 

data  transcription,  102 

development,  47 

editing  data  from,  156-57 

field  administration  of,  95,  96 

materials  processing,  122-25 

number  of  items  in,  344 

quality  control  error  analysis  for,  161 

timing  of,  48 
School  coordinators,  93,  94 

assignment,  94 

responsibilities,  93,  95,  96 
School  districts 

obtaining  cooperation  of,  87-91 
School  Update  Form,  88-90 


School  Worksheet,  93 

materials  processing,  111-12,  113 
Schools 

obtaining  cooperation  of,  87-91 
participation  rates,  for  bridge  samples,  61,  90, 

92;  for  main  samples,  60,  90,  92;  summary, 

90,  92,  344  ^ 
sample  sizes,  for  bridge  samples,  61;  for  main 

samples,  60;  by  session  type,  65 
selection  of,  for  bridge  samples,  60-63;  for  main 

samples,  57-60 
summary  statistics,  344 
Science  data  analysis.  See  Science  trend  da»  ♦ 

analysis 

Science  trend  data  analysis,  329-31,  337-39 
conditioning  effects,  339,  607-9 
conditioning  variables,  339,  555 
equating,  330,  331,  337 
item  analysis,  337 
item  parameter  estimation,  337 
item  parsmet^^rs,  337,  673-77 
items  excluded  from  scaling,  331 
main  objective  of,  330 

mean  proportion  correct  for  item  blocks,  338 

proficiency  estimation,  337 

scaling,  337-39 
Science  trend  samples 

age  definition  for,  330,  332 

booklets  used  for,  330 

modal  grades  for,  332 

mode  of  administration  for,  329,  330 

sample  size,  332 

time  of  testing  for,  330,  332 
Session  Assignment  Form,  63,  91,  95 
Sessions 

summary  statistics  for,  345 

types  of,  60-63 
Size  and  type  of  community  (STOC),  60,  682-83 
Special  study  samples,  7, 17, 180 

age  definition  for,  16 

booklets  used,  16;  contents,  70 

modal  grades  for,  16 

mode  of  adminr^^tration  for,  16 

sample  sizes,  16 

student  cohorts  assessed  for,  16 

time  of  testing  for,  16 
Speededness 

checks  for,  in  item  analysis,  180 

considerations  of,  in  scaling  235 

and  reading  cross-sectional  analysis,  258 
State  Advisory  Committee,  ^3,  44 
State  education  agencies 

and  assessment  development,  42 
State  Testing  Directors,  43 
STOC.  See  Size  and  type  of  community 
Stratification.  See  Primary  sampling  units 
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Student  cohorts 

for  all  student  samples,  16 
for  civics  cross-sectional  samples,  299 
for  civics  trend  samples,  299 
for  geography  cross-sectional  samples,  326 
for  mathematics  trend  samples,  330,  332 
for  reading  cross-sectional  samples,  253 
for  reading  trend  samples,  253 
for  science  trend  samples,  330,  332 
target  populations,  19,  23 
for  US,  history  cross-sectional  samples,  318 
for  U.S.  history  trend  sample,  318 
for  writing  cross-sectional  samples,  268 
for  writing  trend  samples,  268 
3  Student  data 

editing  of,  156 

materials  processing,  115-22 
quality  control  error  analysis  for,  160 

Student  Listing  Form,  93 

Students 

cohorts  assessed  for  each  sample,  16 
distribution  of,  by  age  class  and  sample,  345 
population  estimates  (weighted)  for,  346 
proficiency  estimates  for,  by  subpopulation,  347 
Subject  areas 
assessed  in  1988,  3,  13 
assessed  since  1969, 19-22 
list  of  reports  for,  3 
rationale  for  including,  33 

TBLT  computer  program,  263,  314 
Teacher  data  analysis 

for  reading,  264-65 

for  writing,  295 
Teacher  questionnaire 

changes  in,  8 

contents,  47,  82 

data  editing,  156--57 

data  transcription,  102 

development,  47 

field  administration  of,  95,  96 

materials  proc  ssing,  122-25 

number  of  items  in,  344 

quality  control  error  analysis  for,  161 

summary  statistics,  345 

timing,  48 
Teacher  sample 

selection  of,  for  reading,  67;  for  writing,  67 
Teacher  Survey  Roster,  93 
Technical  report 

audience,  3 

organization  of,  9 

purpose,  3 
TESTFACT  computer  pro-am,  301 
Three-parameter  lo^tic  model 

compu^'mg  plausible  values  for,  237-40 


us^.  in  data  analysis,  181;  for  geography,  327;  for 

reading,  261;  for  U.S.  history,  321 
u  ,e  in  scaling,  232-235 
Tinrx  of  testing,  15 

(or  all  student  samples,  16 
Jiangcs  in,  15 

for  civics  cross-sectional  samples,  299 
for  civics  trend  samples,  299 
effect  of  changes  in,  189 
for  geography  cross-sectional  samples,  326 
for  mathematics  trend  samples,  330,  332 
for  reading  cross-sectional  samples,  253 
for  reading  trend  samples,  253 
for  science  trend  samples,  330,  332 
by  type  of  assessment,  in  sample  design,  52 
for  U.S.  history  cross-sectional  samples,  318 
for  U.S.  history  trend  sample,  318 
for  writing  cross-sectional  samples,  268 
for  writing  trend  samples,  268 
Trend  samples.  See  Bridge  samples;  Civics  trend 

samples;  Mathematics  trend  samples; 

Reading  trend  samples;  Science  trend 

samples;  U.S.  history  trend  sample,  Writing 

trend  samples 

U.S.  history  assessment  development 

background  questions,  45 

items,  44 

objectives,  43 
U.S.  history  cross-sectional  data  analysis,  317-24 

anchoring,  322,  323 

conditioning  effects,  322,  593-98 

conditioning  variables,  322,  545-48;  composite 
and  derived,  628-35 

dichotomization  of  open-ended  items,  319 

item  analysis,  319 

item  calibration,  321 

item  classification,  323-24 

item  response  theory,  321 

items  excluded  from  scaling,  321 

item  parameters,  662^5 

items  used,  317,  321 

KR-20  reliabilities  for  ite^n  blocks,  319,  320 

mean  proportion  correct  for  item  jlocks,  320 

not-reached  perccnts  for  item  blocks,  319 

parameter  estimation,  321 

plausible  values,  322 

proficiency  estimation,  322 

reporting  variables,  composite  and  derived, 

703-4 
scaling,  319,  321-22 
U.S.  history  cross-sectional  samples 
a;e  defmition  for,  318 
bix)klets  used  for,  318 
modal  grades  for,  318 
<;ample  sizes,  318 
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time  of  testing  for,  318 
U.S.  history  data  analysis,  317-24.  See  also  U.S. 
history  cross-sectional  data  analysis;  U.S. 
history  trend  data  analysis 
U.S.  history  trend  data  analysis,  324 

item  analysis,  324 

item  classification,  324 

items  used,  324 
U.S.  history  trend  sample 

age  definition  for,  318 

booklets  used  for,  318 

modal  grades  for,  318 

sample  size,  318 

time  of  testing  for,  318 

Weighting  procedures,  190-207 

adjustments  for  ago-only  eligibjcs  nonresponsc, 

195;  school  nonresponsc,  193;  session 

nonresponse,  194;  student  nonresponsc,  196 
distribution  of  weight  components  for  1988 

samples,  483-96 
for  excluded  student  weights, 
nonresponse  adjustments  for  excluded  student 

wei^ts,  206;  for  student  weights,  192-97;  for 

teacher-student  weights,  207 
poststratification  of  excluded  student  weights, 

206;  of  student  weights,  198-202;  of 

teacher-student  weights,  207 
for  student  base  weight,  191-92 
tape  and  spiral  sample  designations  for,  190 
for  teacher-student  weights,  206 
trimming  of  excluded  student  weights,  206;  of 

student  weights,  197-98;  of  teacher-student 

weights,  207 
Writing  assessment  development 
background  questions,  42 
items,  41 

longer  response  time  for  items,  7,  41,  50; 

booklets  used  to  measure,  70 
objectives,  40 
Writing  cross-sectional  data  analysis,  289-96 
average  response  method,  291-93 
components  of,  289 

conditioning  variables,  291,  523-25;  composite 

and  derived,  294, 615-18 
effect  of  block  position  on  performance,  294 
effect  of  longer  response  time  on  performance, 

295-96 
items  t!sed,  290 
plausible  values,  291 
primary  trait  analysis,  289 
{  -n/^ess  and  instruction  analysis,  294-95 
Pv  ^  )se  ot  267 

reporting  variables,  composite  and  derived, 

696-97 
sample  sizes,  290 
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samples  used,  268 
scaling,  291-93 
teacher  data  analysis,  295 
Writing  cross-sectional  samples 
age  definition  for,  268 
modal  grades  for,  268 
time  of  testing  for,  268 
Writing  data  analysis,  267-96.  See  also  Writing 

cross-sectional  data  analysis;  Writing  trend 

data  analycis 
objectives  of,  267 
Writing  trend  data  analysis,  268-89 

agreement  and  reliability  for  p  imary  trait 

scores,  271-273 
average  response  method  (ARM),  275 
conditioning  variables,  275,  279,  526-28; 

composite  and  derived,  619-20 
effect  of  across-year  variation  Jn  primary  trait 

scoring,  270-75 
holistic  data  analysis,  288,  289 
items  used,  268,  269,  288,  289 
plausible  values,  275-278,  288 
primary  trait  analysis  and  scoring,  268 
purpose  of,  267 

reporting  variables,  composite  and  derived, 
697-99 

rescoring  of  1984  writing  papers,  270 
sample  sizes,  for  holistic  analysis,  289;  for 

mechanics  analysis,  288;  for  primary  trait 

analysis,  274 
samples  used,  268 
scaling,  287-88 
Writing  trend  samples 
age  definition  for,  268 
modal  grades  for,  268 
time  of  testing  for,  268 
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